XEIZE
/Vulnerability documentation/IaC/CloudFormation/AWS/MSK broker log export needs review

MSK broker log export needs review

Collect MSK broker logs at the required destination and verify delivery.

Description

Without retained MSK broker logs, broker errors and connection problems are harder to investigate afterward. Exporting logs to CloudWatch Logs, S3 or Firehose provides records for operational analysis.

Broker logs are not a complete audit of every message or user action. They also serve a different purpose from CloudTrail records of MSK management API calls.

Potential impact

  • Finding the cause of broker failures or processing delays can take longer.
  • Missing operational records may prevent investigation or retention requirements from being met.

Remediation

Enable the required log destination under LoggingInfo.BrokerLogs. Prepare the destination and delivery permissions, then verify collection. Manage sensitive log content, access permissions, retention and costs.

Examples

The old Kafka version and subnet values are examples. Use a currently supported version and broker type with actual subnets. BrokerLogGroupName is the name of a prepared CloudWatch log group; delivery permissions are also required.

Before

yaml
AWSTemplateFormatVersion: "2010-09-09"
Description: MSK Cluster with required properties.
Resources:
  TestCluster5:
    Type: "AWS::MSK::Cluster"
    Properties:
      ClusterName: ClusterWithRequiredProperties
      KafkaVersion: 2.2.1
      NumberOfBrokerNodes: 3
      BrokerNodeGroupInfo:
        InstanceType: kafka.m5.large
        ClientSubnets:
          - ReplaceWithSubnetId1
          - ReplaceWithSubnetId2
          - ReplaceWithSubnetId3

No broker log export is explicit. Check other configuration and operational requirements.

After

yaml
AWSTemplateFormatVersion: "2010-09-09"
Description: MSK Cluster with required properties.
Resources:
  TestCluster5:
    Type: "AWS::MSK::Cluster"
    Properties:
      ClusterName: ClusterWithRequiredProperties
      KafkaVersion: 2.2.1
      LoggingInfo:
        BrokerLogs:
          CloudWatchLogs:
            Enabled: true
            LogGroup: !Ref BrokerLogGroupName
      NumberOfBrokerNodes: 3
      BrokerNodeGroupInfo:
        InstanceType: kafka.m5.large
        ClientSubnets:
          - ReplaceWithSubnetId1
          - ReplaceWithSubnetId2
          - ReplaceWithSubnetId3

The same cluster exports broker logs to CloudWatch Logs. Verify that logs arrive at the destination after configuration.

References

XEIZE DocumentationTechnical support