LearningTree · AWS · Integration

Integration — Other Services
Step Functions · MQ · AppSync · EventBridge Pipes

Four powerful integration services that complete your AWS architecture toolkit: Step Functions for workflow orchestration, Amazon MQ for legacy migration, AppSync for GraphQL APIs, and EventBridge Pipes for point-to-point routing.

⚡ Integration Other Services — 30 Seconds

Step Functions — visual state machine orchestration for multi-step, long-running workflows with error handling and retries
Amazon MQ — managed ActiveMQ/RabbitMQ broker; migrate on-premises JMS/AMQP workloads to AWS without rewriting code
AWS AppSync — fully managed GraphQL API that aggregates multiple AWS data sources into a single endpoint with real-time subscriptions
EventBridge Pipes — point-to-point integration that connects a single source to a single target with optional filtering and enrichment — no broker needed

Chapter One

AWS Step Functions

What is AWS Step Functions? Introductory

When a business process spans multiple services — validate input, call an API, wait for human approval, retry on failure, branch on conditions — you need orchestration, not just messaging. AWS Step Functions is a fully managed serverless orchestration service that lets you coordinate distributed applications and microservices using visual state machines.

👉 Mental model: a recipe with error handling. A recipe lists steps in order, branches ("if gluten-free, skip step 3"), waits ("rest the dough for 1 hour"), repeats ("fold 6 times"), and has fallbacks ("if oven fails, call chef"). Step Functions externalises this logic into a state machine — your Lambda functions only do the individual steps; Step Functions handles the orchestration.

Core Concepts — States and the State Machine Core

Step Functions uses Amazon States Language (ASL), a JSON-based DSL that defines your workflow. A workflow (called a state machine) is a collection of states:

⚙️

Task

Calls an AWS service or activity. Most common state. Invoke Lambda, call DynamoDB, start an ECS task, call 300+ service integrations.

🔀

Choice

Branches based on a condition. Like an if/else. Routes workflow to different states based on input data fields.

⏱️

Wait

Pauses execution for a fixed duration or until a specific timestamp. Perfect for rate-limiting or scheduled delays.

🔄

Parallel

Runs two or more branches simultaneously. All branches must complete before the workflow continues. Great for fan-out.

🗂️

Map

Iterates over an array and runs a sub-workflow for each item — in parallel or sequentially. Process a batch of items.

✅

Pass / Succeed / Fail

Pass transforms data. Succeed ends successfully. Fail ends with an error and message.

State Machine Diagram — Order Fulfilment Workflow Core

Step Functions — Order Fulfilment State Machine

Standard Workflows vs Express Workflows Core

Step Functions offers two workflow types with very different characteristics:

Feature	Standard Workflows	Express Workflows
Max duration	1 year	5 minutes
Execution semantics	Exactly-once	At-least-once
Execution rate	2,000 / second	100,000 / second
Pricing	Per state transition	Per duration + memory
Execution history	Stored in Step Functions (90 days)	CloudWatch Logs only
Audit / replay	Full audit trail in console	No built-in — logs only
Best for	Long-running workflows, human approvals, idempotent business processes	High-volume event processing, IoT, streaming, short burst tasks

👉 Rule of thumb: Use Standard for anything that touches money, orders, or approvals — you need exactly-once and audit history. Use Express for high-volume, sub-5-minute fan-out processing where at-least-once is acceptable.

Built-in Error Handling — Retry and Catch Core

Step Functions has first-class error handling embedded directly in the state definition — no try/catch in your Lambda code:

🔁

Retry

Automatically retry a failed state with exponential back-off:

ErrorEquals — which errors trigger retry
IntervalSeconds — initial wait between retries
BackoffRate — multiply wait by this on each retry
MaxAttempts — stop retrying after this count
Example: retry Lambda throttles 3× with 2s back-off

🪤

Catch

Route to a different state when retries are exhausted:

ErrorEquals — which error class to catch
Next — state to transition to on error
ResultPath — where to put the error object in the input
Example: catch payment failure → go to "RefundAndNotify" state

Service Integrations — 300+ Native Connections Core

Step Functions can directly call AWS services without Lambda acting as a proxy. Two integration patterns:

⚡

Request-Response (Default)

Call the service and move on immediately — don't wait for the job to finish. Use for fire-and-forget: send an SNS message, write a DynamoDB item, start an ECS task.

⏳

Sync / Wait-for-Callback (.sync, .waitForTaskToken)

.sync: Wait for the job to complete (e.g., Glue job, ECS task, SageMaker training). .waitForTaskToken: Pause indefinitely until an external system calls back with a token — perfect for human approval workflows.

Service	Integration Pattern	Use Case
Lambda	Request-Response / Sync	Custom business logic
DynamoDB	Request-Response	Read/write state without Lambda
ECS / Fargate	Sync	Container tasks — wait for completion
Glue	Sync	ETL jobs — wait for result
SageMaker	Sync	Training/processing — wait for model
SNS / SQS	Request-Response	Notify or enqueue without Lambda hop
API Gateway / HTTP	Request-Response	Call external REST APIs
Human Approval	waitForTaskToken	Email → approver clicks link → resume

Common Architecture Patterns Core

🛒

E-commerce Order Pipeline

Validate → Charge → Reserve Inventory → Notify Shipping → Wait for Delivery → Send Review Request. Long-running (days), exactly-once semantics required.

🖼️

Batch Media Processing

Map state iterates over uploaded images: resize × 3 sizes in parallel per image. High-volume Express workflow. Returns when all sizes are ready.

✅

Human Approval Workflow

Submit expense → waitForTaskToken → email manager with approve/reject link → resume state machine on callback. Manager has up to 7 days to respond.

🤖

ML Pipeline Automation

Trigger Glue job (data prep) → wait for completion → start SageMaker training → wait → evaluate model → conditional deploy or retrain.

🎓 Exam Insight

Long-running multi-step workflow with error handling → Step Functions Standard Workflow
Human approval in an automated process → waitForTaskToken integration pattern
High-volume sub-5-min event processing → Step Functions Express Workflows
Fan-out: process each item in a list concurrently → Map state with concurrency control
Avoid Lambda orchestrating Lambda — Step Functions eliminates "Lambda calling Lambda" anti-pattern; use service integrations instead
vs SWF (Simple Workflow Service): Step Functions is the modern replacement. SWF is legacy — only choose it if you need external signals or need to run code on-premises within the workflow.

👉 Key Takeaway

Step Functions externalises orchestration logic — your Lambda functions do the work, Step Functions decides the flow, handles retries, branches on conditions, and coordinates 300+ AWS services without code.

Chapter Two

Amazon MQ — Managed Message Broker

The Migration Problem That Amazon MQ Solves Introductory

Thousands of enterprise applications use JMS (Java Message Service), AMQP, MQTT, or STOMP protocols to communicate via message brokers like Apache ActiveMQ or RabbitMQ. When these teams want to move to AWS, they face a choice:

⚠️

Option A: Rewrite for SQS/SNS

Months of refactoring work
Change every producer and consumer
Risk introducing new bugs during migration
New protocols — requires retraining teams
Unsuitable for lift-and-shift

✅

Option B: Amazon MQ (Lift and Shift)

Keep existing JMS/AMQP/MQTT/STOMP code
AWS manages broker setup, patching, backups
Just point your app at the new endpoint
Migrate to cloud in days, not months
Focus modernisation effort on business logic

👉 Amazon MQ is not a replacement for SQS/SNS. If you are building a new cloud-native application, use SQS or EventBridge. Amazon MQ exists specifically so that existing applications using open-standard messaging protocols can reach AWS with zero code changes.

Supported Brokers and Protocols Core

Broker Engine	Supported Protocols	Best For
Apache ActiveMQ	OpenWire, AMQP 1.0, MQTT, STOMP, WebSocket	JMS-based Java enterprise apps, Spring JMS templates, complex routing via Virtual Topics
RabbitMQ	AMQP 0-9-1	Exchange-based routing, fanout/topic/direct exchanges, RabbitMQ-native apps

How Amazon MQ Works Core

Amazon MQ — Architecture: On-Premises Migration to AWS

Deployment Configurations Core

🖥️

Single-Instance

One broker in one AZ. Low cost. No HA. Best for dev/test. Downtimes require maintenance windows.

🔄

Active/Standby (ActiveMQ)

Two brokers: active in one AZ, standby in another. EFS shared storage for messages. Automatic failover — clients reconnect. Best for production.

🏗️

Cluster Deployment (RabbitMQ)

Three-node cluster across three AZs. Messages replicated to all nodes. Active-active — any node can handle requests.

Amazon MQ vs SQS/SNS — When to Use Each Core

Factor	Amazon MQ	SQS / SNS
Use case	Migrate existing JMS/AMQP apps	Build new cloud-native apps
Protocols	JMS, AMQP, MQTT, STOMP, OpenWire	AWS SDK (HTTP/REST) only
Scaling	Fixed instance size — manual scaling	Fully elastic, scale to millions/s
Management	You pick instance type; broker config needed	Fully serverless — zero config
Pricing	Per broker-hour (like EC2)	Per request (pay-as-you-go)
Networking	Must be in VPC; private endpoint	AWS-managed endpoints
Code change required?	No — same AMQP/JMS code	Yes — rewrite to use AWS SDK

🎓 Exam Insight

"Migrate on-premises ActiveMQ / RabbitMQ to cloud" → Amazon MQ
"Application uses JMS / AMQP / MQTT" → Amazon MQ (no code change needed)
"Build new messaging for a cloud-native app" → SQS, SNS, or EventBridge (NOT Amazon MQ)
Amazon MQ is NOT serverless — it runs on broker instances; you must choose instance size; does not scale to zero
Multi-AZ HA: ActiveMQ uses active/standby + EFS. RabbitMQ uses 3-node cluster across 3 AZs

👉 Key Takeaway

Amazon MQ is the migration path for legacy message brokers — it lets you move on-premises ActiveMQ and RabbitMQ workloads to AWS without changing a line of application code.

Chapter Three

AWS AppSync — Managed GraphQL

What is AWS AppSync? Introductory

Modern mobile and web apps often need data from multiple backends simultaneously — user profile from a database, their orders from another, real-time order status from a stream. Traditionally this means multiple REST API calls. AWS AppSync is a fully managed GraphQL API service that aggregates multiple data sources into a single flexible endpoint, with built-in real-time pub/sub and offline sync.

❓

The REST Problem

Mobile app needs user + orders + products in one screen. Three round-trip REST calls. Over-fetching. Under-fetching. N+1 queries. Slow on mobile networks.

✅

GraphQL Solution

One query: "give me user, their last 5 orders, and product images." Client gets exactly what it asked for — no more, no less. One network round trip.

⚡

AppSync Addition

Manages the GraphQL layer: resolvers, auth, caching, subscriptions, offline merging. You just define the schema and wire it to data sources.

AppSync Architecture — How It Works Core

AWS AppSync — GraphQL API Aggregating Multiple Data Sources

GraphQL Operations — Query, Mutation, Subscription Core

📥

Query

Read data. The client specifies exactly which fields it needs. AppSync fires resolvers for each requested field and merges results.

query {
  user(id: "u1") {
    name email
    orders(last: 5) {
      id status total
    }
  }
}

✏️

Mutation

Write/update data. After a mutation, AppSync can automatically push the updated data to all subscribed clients via WebSocket.

mutation {
  placeOrder(input: {
    items: [{sku:"A1"}]
    total: 49.99
  }) {
    id status
  }
}

📡

Subscription

Real-time updates over WebSocket. Client subscribes to events (e.g., order status changes). AppSync pushes data as mutations occur.

subscription {
  onOrderUpdate(
    orderId: "o123"
  ) {
    status updatedAt
  }
}

Authorization Models Core

Auth Mode	How It Works	Best For
API Key	Static key in the request header. Max 1 year expiry.	Public-facing or dev/test APIs
Cognito User Pools	JWT token from Cognito. Directives like `@auth(rules: [...])` control field access.	Authenticated user-facing apps
IAM	SigV4 signed requests. Works with IAM roles/users. For machine-to-machine.	Service-to-service, Lambda calling AppSync
OIDC	Any OpenID Connect provider (Auth0, Okta)	Third-party identity providers
Lambda	Custom Lambda function returns allow/deny. Arbitrary logic.	Complex custom auth requirements

Resolvers — Connecting Schema Fields to Data In-Depth

📋

Unit Resolvers

One resolver → one data source. Written in VTL (Velocity Template Language) or JavaScript. Maps GraphQL field to DynamoDB query, HTTP call, or Lambda invocation. Simple 1-to-1 mapping.

🔗

Pipeline Resolvers

Chain multiple Functions (each calling a data source) in sequence. Output of one function becomes input of the next. For complex queries that touch multiple sources before returning a result.

Offline Support and Conflict Resolution In-Depth

📵

Offline Data Sync

AppSync (with AWS Amplify DataStore) supports offline-first apps. Data is written locally when offline. When connectivity returns, changes sync with the backend automatically.

⚔️

Conflict Detection & Resolution

When offline changes conflict with server changes, AppSync supports three strategies: Optimistic Concurrency (version check), Auto Merge (field-level merge), or Lambda (custom logic).

🎓 Exam Insight

"Mobile app needs real-time updates without polling" → AppSync subscriptions over WebSocket
"Aggregate data from DynamoDB, Lambda, and RDS in one API call" → AppSync
"Offline-first mobile app with sync" → AppSync + Amplify DataStore
"GraphQL vs REST on AWS" → AppSync is managed GraphQL; API Gateway is managed REST/HTTP
AppSync caching: You can enable server-side caching per resolver (TTL from 1 second to 3600 seconds) to reduce data source load

👉 Key Takeaway

AppSync turns your scattered data sources into a single flexible GraphQL endpoint — with real-time subscriptions, offline sync, and fine-grained auth control — so clients get exactly the data they need in one request.

Chapter Four

EventBridge Pipes — Point-to-Point Routing

What is EventBridge Pipes? Introductory

Before Pipes, connecting an SQS queue to a Step Functions workflow required writing Lambda glue code — read from queue, transform the message, start execution. Amazon EventBridge Pipes eliminates this glue code by providing a fully managed, point-to-point integration with a built-in pipeline: Source → Filter → Enrichment → Target.

👉 Key distinction from EventBridge bus: The EventBridge bus is a many-to-many event router — many sources, many rules, many targets. EventBridge Pipes is one-to-one — one source, one target, with optional filtering and enrichment in between. No bus, no rules, no fan-out.

The Four Stages of a Pipe Core

EventBridge Pipes — Four-Stage Pipeline: Source → Filter → Enrich → Target

Supported Sources Core

🗂️

Queue Sources

Amazon SQS (Standard & FIFO)
Amazon MQ — ActiveMQ
Amazon MQ — RabbitMQ

Pipes polls the queue; you control batch size.

🌊

Stream Sources

Amazon Kinesis Data Streams
Amazon DynamoDB Streams

Pipes consumes records in order per shard; supports starting position.

📨

Messaging Sources

Apache Kafka (MSK)
Self-managed Kafka

Consumer group semantics; topic + partition.

Real-World Pattern — SQS to Step Functions without Lambda Glue Core

EventBridge Pipes — Eliminating Lambda Glue Code

EventBridge Pipes vs EventBridge Bus — Key Differences Core

Feature	EventBridge Bus	EventBridge Pipes
Topology	Many-to-many: multiple sources, rules, multiple targets	One-to-one: single source → single target
Sources	AWS services (push model) + PutEvents API	Polling-based (SQS, Kinesis, DynamoDB Streams, MQ, Kafka)
Fan-out	Yes — one rule, up to 5 targets	No — single target per pipe
Filtering	Event rules with rich pattern matching	Per-pipe filter before target
Enrichment step	No — input transformers only	Yes — Lambda/Step Functions/API GW enrichment step
Use case	Event routing hub, complex routing logic	Simple P2P glue code elimination
Batching	Not applicable (single event model)	Yes — batch size + concurrency configured per pipe

When to Use Pipes vs Bus Core

🔀

Use EventBridge Bus when…

Multiple consumers care about the same event
Complex routing rules needed (content-based)
AWS service events (EC2, S3, GuardDuty)
Cross-account event distribution
Scheduled/cron-based triggers

🔗

Use EventBridge Pipes when…

Simple source → target integration (1:1)
Polling-based source (SQS, Kinesis, DynamoDB Streams)
Would otherwise write Lambda "glue" code
Need per-event enrichment before delivering
Connecting MQ or Kafka to a downstream target

🎓 Exam Insight

"Connect SQS/Kinesis/DynamoDB Streams to a target without Lambda glue" → EventBridge Pipes
"Filter DynamoDB Stream changes before sending to Lambda" → EventBridge Pipes with filter stage
Pipes vs Lambda EventSourceMapping: Both can consume SQS/Kinesis. Pipes adds the optional filter and enrichment steps — no Lambda code needed for simple routing. EventSourceMapping is still valid for direct Lambda invocation.
Enrichment vs Transformation: Enrichment is a full AWS service call (adds new data). Transformation (input transformer) just reshapes the existing data.

👉 Key Takeaway

EventBridge Pipes replaces Lambda glue code for polling-based sources — it connects one source to one target with optional filtering and enrichment, fully managed and without writing infrastructure code.

Chapter Five

Choosing the Right Integration Service

Full Service Comparison Core

Service	Primary Purpose	Protocol/Interface	Scale Model	When NOT to Use
Step Functions	Multi-step workflow orchestration	AWS SDK, SDK integrations	Serverless (pay per transition)	Simple single-step tasks; high-volume sub-second processing (too expensive)
Amazon MQ	Managed AMQP/JMS broker	AMQP, JMS, MQTT, STOMP	Instance-based (fixed capacity)	New cloud-native apps; need elastic scale; no legacy broker dependency
AWS AppSync	Managed GraphQL API	GraphQL over HTTP + WebSocket	Serverless (pay per request)	Simple REST APIs; no real-time or multi-source data needs
EventBridge Pipes	P2P event routing (source → target)	AWS SDK / polling	Serverless (pay per invocation)	Fan-out to multiple targets; push-based sources (use EventBridge bus instead)
EventBridge Bus	Many-to-many event routing	AWS SDK / AWS service events	Serverless (pay per event)	Polling-based sources; simple P2P (Pipes is simpler)
SQS	Durable message queue	AWS SDK (HTTP)	Serverless, unlimited scale	Fan-out to many consumers; need to route based on content

Decision Guide — Which Service to Pick? Core

🔧

Use Step Functions when…

Workflow spans multiple services over time
Need retries, error handling at the orchestration level
Human approval or external callback required
Fan-out processing where all branches must complete (Parallel)
Long-running (minutes to months)

🐇

Use Amazon MQ when…

Migrating on-premises ActiveMQ or RabbitMQ
App uses JMS, AMQP, MQTT, STOMP protocols
Cannot change application code during migration
Virtual Topics, Message Selectors needed (ActiveMQ-specific)
RabbitMQ exchange routing required

📊

Use AppSync when…

Mobile or web app needs data from multiple sources in one call
Real-time updates without polling (subscriptions)
Offline-first with automatic sync (Amplify DataStore)
Fine-grained field-level authorization
Aggregating DynamoDB + Lambda + RDS into one API

🔗

Use EventBridge Pipes when…

Connecting polling-based source (SQS, Kinesis, DynamoDB Streams) to a target
Simple 1:1 integration — no fan-out needed
Want to avoid writing Lambda glue code
Need filter + optional enrichment before target
Bridging Amazon MQ / Kafka into AWS services

Combined Architecture — All Services Working Together In-Depth

Modern E-commerce Architecture — All Integration Services in Play

Exam Quick Reference Core

Scenario in Exam Question	Answer
Multi-step workflow with error handling, retries, and conditional branching	Step Functions
Manager must approve a request before the workflow continues	Step Functions — `waitForTaskToken`
High-volume short-lived event processing (IoT, streaming) — Step Functions applicable?	Step Functions Express Workflow (not Standard)
Migrate on-premises ActiveMQ to AWS without changing application code	Amazon MQ
Build a new cloud-native queuing system	SQS, NOT Amazon MQ
Mobile app needs real-time updates when data changes	AppSync subscriptions over WebSocket
Aggregate DynamoDB + RDS + external API in one GraphQL call	AppSync
Connect DynamoDB Streams to Step Functions without a Lambda function	EventBridge Pipes
One event must fan-out to many different services	EventBridge Bus (not Pipes)

🎯 Domain Summary — Integration: Other Services

Step Functions — Serverless state machine that orchestrates multi-step workflows with error handling, branching, waiting, and 300+ native AWS service integrations
Standard vs Express: Standard = exactly-once, 1 year max, audit trail. Express = at-least-once, 5 min max, high throughput for events
Amazon MQ — Managed ActiveMQ and RabbitMQ; the right choice when migrating JMS/AMQP applications — not for new cloud-native apps
AppSync — Managed GraphQL that aggregates multiple data sources, supports real-time WebSocket subscriptions, and offline sync for mobile apps
EventBridge Pipes — Point-to-point, source→filter→enrich→target pipeline; eliminates Lambda glue code for polling-based sources (SQS, Kinesis, DynamoDB Streams, MQ, Kafka)
Pipes vs Bus: Pipes = 1:1. Bus = many-to-many with rules and fan-out