Understanding Push vs. Pull in AWS Event-Driven Architecture

• Push vs. Pull in AWS EDA: A Quick Overview
  • Push-Based in EDA
  • Pull-Based in EDA
• The Magic of Event Source Mapping (ESM) for Pull Sources
  • Turning Pull into “Push” Internally
• How ESM Can Be Obfuscated in AWS SAM Templates
• Why Push vs. Pull Matters for Better Decoupling
• Conclusion

Push vs. Pull in AWS EDA: A Quick Overview

Push-Based in EDA

A push-based mechanism in AWS means the event publisher actively pushes data or notifications to the event consumer.

• Examples:
  • Amazon Simple Notification Service (SNS): SNS sends messages to subscribed endpoints (like Lambda functions, HTTP endpoints, emails) the instant an event is published.
  • Amazon API Gateway: When an API request is made, it triggers your Lambda function immediately—essentially “pushing” the request to your function.

The big win here is simplicity. You write your consumer logic in a Lambda function and then configure SNS or API Gateway to call it. There’s no custom polling logic needed.

Pull-Based in EDA

A pull-based mechanism is where the consumer polls or pulls messages from the event source.

• Example:
  • Amazon Simple Queue Service (SQS): Messages sit in an SQS queue until a consumer (i.e., a Lambda function or custom app) actively retrieves (polls) them.

This model typically adds an extra layer of control: you can manage concurrency, back-off, and error handling more granularly. However, it can be a bit more complex to implement because you have to manage the polling aspect—or so it seems until you consider AWS Lambda’s Event Source Mapping magic.

The Magic of Event Source Mapping (ESM) for Pull Sources

Event Source Mapping is a feature in AWS Lambda that natively takes care of polling for you when integrating with services like SQS, DynamoDB Streams, and Kinesis. In short, the Lambda service becomes the poller, pulling messages from the queue/stream and invoking your function automatically.

Official AWS Doc – https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-lambda-eventsourcemapping.html

Turning Pull into “Push” Internally

Though SQS is fundamentally pull-based, you can almost treat it like it’s push-based once you set up Event Source Mapping:

1. You configure your Lambda function to read from an SQS queue.
2. Under the hood, AWS “pulls” messages from SQS for you and immediately “pushes” them to your Lambda function.
3. Your Lambda code never has to handle the actual polling logic.

This approach merges the best of both worlds. You benefit from SQS decoupling and control, but your Lambda function behaves like it’s getting events pushed to it in real time.

How ESM Can Be Obfuscated in AWS SAM Templates

When using AWS Serverless Application Model (SAM), you often just declare something like:

MyFunction:
  Type: AWS::Serverless::Function
  Properties:
    CodeUri: ./src/
    Handler: app.handler
    Runtime: python3.9
    Events:
      MyQueueEvent:
        Type: SQS
        Properties:
          Queue: !GetAtt MyQueue.Arn

Notice there’s no explicit “Event Source Mapping” object. SAM transforms that Events section into the appropriate AWS CloudFormation resources under the hood. ESM creation is handled automatically, so you might not even realize that behind the scenes it’s building all the poller logic that turns SQS from a pull-based service into a near-instant push mechanism to your Lambda.

This obfuscation is great for productivity (and easier code!), but it’s essential to know what’s happening backstage—especially if you’re troubleshooting performance, concurrency, or scaling concerns.

If you want to be explicit, here’s an example of an AWS::Serverless::EventSourceMapping resource in a SAM template:

Resources:
  MyFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: ./src/
      Handler: app.handler
      Runtime: python3.9
      # ... other function properties

  MyEventSourceMapping:
    Type: AWS::Serverless::EventSourceMapping
    Properties:
      FunctionName: !GetAtt MyFunction.Arn
      EventSourceArn: !GetAtt MyQueue.Arn
      BatchSize: 2

Why Push vs. Pull Matters for Better Decoupling

• Scalability & Concurrency: Push-based models like SNS → Lambda scale quickly and automatically. Pull-based models with ESM let you fine-tune concurrency limits and manage traffic bursts more gracefully.
• Reliability & Control: Pull-based queues buffer events, so even if your consumer is down or throttled, messages don’t get lost. You can control the polling rate, the concurrency, and your retry logic.
• Decoupling: The fundamental reason for EDA is to remove tight coupling. Push-based or pull-based, as long as you’re asynchronous, you’re freeing up your architecture from point-to-point dependencies.

In AWS, you often end up combining both:

• SNS → SQS → Lambda for a layered approach where SNS fans out messages to SQS queues for different consumers. Each consumer’s Lambda function uses ESM to poll its queue on its own terms.

Conclusion

Event-driven architecture is a powerful pattern for building modern, scalable, and resilient applications in AWS. Understanding the differences between push and pull mechanisms and the role of Event Source Mapping in AWS Lambda is crucial for choosing the right approach for your specific use case. By leveraging the appropriate mechanisms and services, you can build highly decoupled and efficient applications that can respond to events in real-time and adapt to changing business needs.