CloudWatch monitoring for unauthorized API calls needs review

Description

Counting CloudTrail authorization errors in CloudWatch Logs and raising an alarm can help operators investigate repeated failures or suspicious calls. A missing filter or an alarm linked to the wrong metric can prevent useful warnings. UnauthorizedOperation and AccessDenied errors can also result from application permission mistakes; an error alone does not establish a compromise.

Potential impact

Repeated unauthorized calls or application permission problems can take longer to identify.
Even when the alarm evaluates correctly, an incorrect notification destination or subscription can prevent the responsible team from receiving a warning.

Remediation

Verify that the required CloudTrail events from the relevant accounts and Regions reach the intended log group. Configure a filter for *UnauthorizedOperation or AccessDenied* errors and test it against the actual log format.
Match the alarm's metric name, namespace, and any required dimensions to the filter's metric_transformation. Set evaluation periods, statistics, thresholds, and missing-data handling for your operational needs.
Set alarm_actions to actual destination ARNs and check alarm actions, SNS permissions, and subscription status. Verify that newly ingested test events produce metric values and trigger the alarm and notification when evaluation conditions are met. Metric filters do not process logs from before their creation retroactively.

Examples

These partial examples show the connection between a filter and an alarm. Define the referenced log group and SNS topic and configure CloudTrail log delivery before using them.

Before

hcl

resource "aws_cloudwatch_metric_alarm" "cis_unauthorized_api_calls_cw_alarm" {
  alarm_name                = "CIS-3.1-UnauthorizedAPICalls"
  comparison_operator       = "GreaterThanOrEqualToThreshold"
  evaluation_periods        = "1"
  metric_name               = "XXXX NOT YOUR FILTER XXXX"
  namespace                 = "CIS_Metric_Alarm_Namespace"
  period                    = "300"
  statistic                 = "Sum"
  threshold                 = "1"
  alarm_description         = "Monitoring unauthorized API calls will help reveal application errors and may reduce time to detect malicious activity."
  alarm_actions             = [aws_sns_topic.CIS_Alerts_SNS_Topic.arn]
  insufficient_data_actions = []
}

resource "aws_cloudwatch_log_metric_filter" "cis_unauthorized_api_calls_metric_filter" {
  name           = "CIS-UnauthorizedAPICalls"
  pattern        = "{ ($.errorCode = \"*UnauthorizedOperation\") || ($.errorCode = \"AccessDenied*\") }"
  log_group_name = aws_cloudwatch_log_group.CIS_CloudWatch_LogsGroup.name

  metric_transformation {
    name      = "CIS-UnauthorizedAPICalls"
    namespace = "CIS_Metric_Alarm_Namespace"
    value     = "1"
  }
}

After

hcl

resource "aws_cloudwatch_metric_alarm" "cis_unauthorized_api_calls_cw_alarm" {
  alarm_name                = "CIS-3.1-UnauthorizedAPICalls"
  comparison_operator       = "GreaterThanOrEqualToThreshold"
  evaluation_periods        = "1"
  metric_name               = "CIS-UnauthorizedAPICalls"
  namespace                 = "CIS_Metric_Alarm_Namespace"
  period                    = "300"
  statistic                 = "Sum"
  threshold                 = "1"
  alarm_description         = "Monitoring unauthorized API calls will help reveal application errors and may reduce time to detect malicious activity."
  alarm_actions             = [aws_sns_topic.CIS_Alerts_SNS_Topic.arn]
  insufficient_data_actions = []
}

resource "aws_cloudwatch_log_metric_filter" "cis_unauthorized_api_calls_metric_filter" {
  name           = "CIS-UnauthorizedAPICalls"
  pattern        = "{ ($.errorCode = \"*UnauthorizedOperation\") || ($.errorCode = \"AccessDenied*\") }"
  log_group_name = aws_cloudwatch_log_group.CIS_CloudWatch_LogsGroup.name

  metric_transformation {
    name      = "CIS-UnauthorizedAPICalls"
    namespace = "CIS_Metric_Alarm_Namespace"
    value     = "1"
  }
}

Explanation:

Before: metric_name differs from the name published by the filter, so the alarm does not evaluate that error metric.
After: The metric name and namespace match the filter's output, and the action references the SNS topic's actual ARN. Verify alarm evaluation and notification receipt after deployment.