AWS Log Subscription Filter: A Hands-On Guide with AWS SAM

May 19, 2024 One Comment

Introduction

In this hands-on tutorial, we’ll dive into creating an AWS Log Subscription Filter using AWS SAM (Serverless Application Model). This project is a practical demonstration of how to capture specific events from your CloudWatch Logs, process them with a Java-based AWS Lambda function, and send targeted notifications to an SNS topic when relevant log entries are detected.

By the end of this guide, you’ll be equipped with the knowledge to set up AWS resources using SAM, write a Lambda function to handle log events and create custom filters to focus on the logs that matter most to you.

Project Setup

Before we start, make sure you have the following prerequisites in place:

  • IntelliJ IDE
  • An active AWS account
  • AWS SAM CLI installed
  • Maven installed
  • Basic familiarity with Java and core AWS services

Installing the AWS Toolkit plugin in IntelliJ IDE

Once the plugin is installed please go through the below official documentation of AWS to get yourself familiarized with creating/updating/deleting SAM templates.

https://docs.aws.amazon.com/toolkit-for-jetbrains/latest/userguide/sam.html

This method ensures you can do this at your personal AWS account without the need to have an AWS Code pipeline/Jenkins pipeline or any other CI/CD tools.

Crafting the AWS SAM Template

Our first step involves creating an AWS SAM template (in YAML) to define the required AWS resources. This includes our Lambda functions, an SNS topic for notifications, and a CloudWatch log group to capture the logs we’re interested in.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: >
  LogSubGenApp

  Sample SAM Template for LogSubGenApp

# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
  Function:
    Timeout: 20
    MemorySize: 128
    Tags:
      Publisher: AWS
      Department: IT
      CostCenter: "12345"
      Owner: "PBKN"
      Project: "LogSubGenApp"

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: HelloWorldFunction
      Handler: helloworld.App::handleRequest
      Runtime: java8.al2
      Architectures:
        - x86_64
      MemorySize: 512
      Environment: # More info about Env Vars: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#environment-object
        Variables:
          PARAM1: VALUE
          JAVA_TOOL_OPTIONS: -XX:+TieredCompilation -XX:TieredStopAtLevel=1 # More info about tiered compilation https://aws.amazon.com/blogs/compute/optimizing-aws-lambda-function-performance-for-java/

  HelloWorldFunctionLogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Sub "/aws/lambda/${HelloWorldFunction}"
      RetentionInDays: 7

  ErrorFunctionRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: "ErrorFunction-Role"
      Path: "/"
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole
        - arn:aws:iam::aws:policy/AmazonSNSFullAccess
      AssumeRolePolicyDocument:
        Version: "2012-10-17"
        Statement:
          - Effect: "Allow"
            Principal:
              Service:
                - "lambda.amazonaws.com"
            Action:
              - "sts:AssumeRole"

  ErrorFunction:
    Type: AWS::Serverless::Function # More info about Function Resource: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#awsserverlessfunction
    Properties:
      CodeUri: ErrorFunction
      Handler: error.ErrorNotif::handleRequest
      Role:  !GetAtt ErrorFunctionRole.Arn
      Runtime: java8.al2
      Architectures:
        - x86_64
      MemorySize: 512
      Environment: # More info about Env Vars: https://github.com/awslabs/serverless-application-model/blob/master/versions/2016-10-31.md#environment-object
        Variables:
          PARAM1: VALUE
          JAVA_TOOL_OPTIONS: -XX:+TieredCompilation -XX:TieredStopAtLevel=1 # More info about tiered compilation https://aws.amazon.com/blogs/compute/optimizing-aws-lambda-function-performance-for-java/
      Events: #LogSubscription filter will be automatically added by SAM using this event
        CWLog:
          Type: CloudWatchLogs
          Properties:
            LogGroupName:
              Ref: HelloWorldFunctionLogGroup
            FilterPattern: '?Error ?error ?Fail ?fail ?Invalid ?invalid ?Exception ?exception'

  ErrorFunctionLogGroup:
    Type: AWS::Logs::LogGroup
    Properties:
      LogGroupName: !Sub "/aws/lambda/${ErrorFunction}"
      RetentionInDays: 7

Outputs:
  HelloWorldFunction:
    Description: "Hello World Lambda Function ARN"
    Value: !GetAtt HelloWorldFunction.Arn
  HelloWorldFunctionIamRole:
    Description: "Implicit IAM Role created for Hello World function"
    Value: !GetAtt HelloWorldFunctionRole.Arn
  ErrorFunction:
    Description: "Error Lambda Function ARN"
    Value: !GetAtt ErrorFunction.Arn
  ErrorFunctionIamRole:
    Description: "Explicit IAM Role created for Error function"
    Value: !GetAtt ErrorFunctionRole.Arn

AWS Lambda Function to generate Logs

Since this is a POC, we are going to create a Hello World Function which has the only purpose to just log whatever input we send it as event params.

/**
 * Handler for requests to Lambda function.
 */
public class App implements RequestHandler<Object, Object> {

    private static final Logger logger = LogManager.getLogger(App.class);

    public Object handleRequest(final Object input, final Context context) {
        logger.info(input.toString());
        return "SUCCESS";
    }

}

Creating an SNS Topic and Subscribing to an EMAIL

Although SNS Topic can be created as part of SAM creation. We are explicitly creating it to make it more robust and standalone. So that, any number of applications can centrally use it for Error Log notification.

(If you want to automate that process, refer to – Auto-Healing SNS Subscriptions in AWS SAM: A Solution for Large Enterprises)

Handling Log Events in Your Error Notification Lambda Function

The heart of our log processing logic resides within the ErrorNotif Lambda function. This function is responsible for decoding, decompressing, and parsing incoming log events. It then applies your custom filters to identify specific log entries and sends notifications through the SNS topic when a match is found.

Here’s a breakdown of the ErrorNotif function’s core tasks:

  1. Extracts raw log data from the AWS event input.
  2. Decodes and decompresses the log data.
  3. Parses the log events from JSON format.
  4. Filters out log entries based on your exclusion criteria.
  5. Processes the remaining relevant log events and triggers SNS notifications.
public class ErrorNotif implements RequestHandler<Map<String, Object>, String> {

    private static final Logger logger = LogManager.getLogger(ErrorNotif.class);

    @Override
    public String handleRequest(Map<String, Object> input, Context context) {
        // Extract the data field
        Map<String, Object> awsLogs = (Map<String, Object>) input.get("awslogs");
        String encodedData = (String) awsLogs.get("data");

        try {
            // Decode the data from Base64
            byte[] decodedData = Base64.decodeBase64(encodedData);

            // Decompress the data using GZIP
            String decompressedData = decompressGzip(decodedData);
            logger.info("Decompressed data: {}", decompressedData);

            // Parse the JSON log events
            Map<String, Object> logEvent = new ObjectMapper().readValue(decompressedData, HashMap.class);
            List<Map<String, Object>> logEvents = (List<Map<String, Object>>) logEvent.get("logEvents");

            // Process each log event
            for (Map<String, Object> event : logEvents) {
                String message = (String) event.get("message");

                // Check for exclusion phrases
                if (message.contains("Error Connection") || message.contains("Failed Connection")) {
                    logger.info("Log message excluded due to containing known error phrases.");
                    continue; // Skip this log
                }

                // Process the log event
                processLogMessage(message);
            }
        } catch (IOException e) {
            logger.error("Error processing log data", e);
        }

        return "SUCCESS";
    }

    private void processLogMessage(String message) {
        try (SnsClient snsClient = SnsClient.create()) {
            String errorMsg = "Something unexpected happened! Go check your system";
            errorMsg = errorMsg.concat(System.lineSeparator());
            errorMsg = errorMsg.concat("Go to https://ap-south-1.console.aws.amazon.com/console/home?region=ap-south-1#");
            String topicArn = "arn:aws:sns:ap-south-1:401135408972:ErrorNotification"; // Already created and subscribed
            pubTopic(snsClient, errorMsg, topicArn);
        }
    }

    public static void pubTopic(SnsClient snsClient, String message, String topicArn) {
        try {
            PublishRequest request = PublishRequest.builder().message(message).subject("Error / Exception occurred in your AWS SAM App") // Added explicitly for email subscribers
                    .topicArn(topicArn).build();

            PublishResponse result = snsClient.publish(request);
            logger.info("{} Message sent. Status is {}", result.messageId(), result.sdkHttpResponse().statusCode());

        } catch (SnsException e) {
            logger.error(e.awsErrorDetails().errorMessage());
        }
    }

    private String decompressGzip(byte[] compressedData) throws IOException {
        try (GZIPInputStream gis = new GZIPInputStream(new ByteArrayInputStream(compressedData)); InputStreamReader reader = new InputStreamReader(gis); BufferedReader in = new BufferedReader(reader)) {

            StringBuilder decompressedData = new StringBuilder();
            String line;
            while ((line = in.readLine()) != null) {
                decompressedData.append(line);
            }
            return decompressedData.toString();
        }
    }
}

Finally, verify your app

Try triggering your Hello World Function to generate different use cases of logs to verify whether it is triggering logs containing “Error”, “error”, “Invalid”, “invalid”, “Exception”, “exception”, “Fail”, “fail” correctly and also to check whether it is correctly excluding logs containing “Error Connection” or “Failed Connection” as expected.

Conclusion

Full version of code available on GitHub – https://github.com/pbkn/LogSubGenApp/tree/main

Congratulations! You’ve successfully implemented a log subscription filter using AWS SAM. We covered setting up AWS resources, crafting a Java-based Lambda function for log processing, and applying custom filters to pinpoint specific log events. This solution is a powerful foundation for building more sophisticated log monitoring and analysis workflows in your AWS environment.

By combining the capabilities of AWS SAM and Lambda, you’re well on your way to developing scalable and efficient serverless applications that react to log events in real-time.

CategoriesAWS
Tags, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,