AWS · Data & Analytics
Event-Driven Data Ingestion Pipeline
High-throughput event streaming with API Gateway, EventBridge, SNS, SQS, Lambda workers, DynamoDB, and S3 Data Lake. This diagram is written in ArchLex and passes AWS provider checks.
Diagram and source
provider aws
direction LR
validation normal
ingress_api: api-gateway
ingest_fn: lambda
events_bus: eventbridge
notification_topic: sns
buffer_queue: sqs
processor_fn: lambda
lake: s3
nosql_store: dynamodb
ingress_api -[invokes]-> ingest_fn
ingest_fn -[publishes]-> notification_topic
notification_topic -[publishes]->|forwards| buffer_queue
events_bus -[triggers]-> buffer_queue
buffer_queue -[invokes]->|batch| processor_fn
processor_fn -[writes]-> nosql_store
processor_fn -[archives]-> lakeWhat is in this diagram.
Resources
- ingress_api Amazon API Gateway
- ingest_fn AWS Lambda
- events_bus Amazon EventBridge
- notification_topic Amazon SNS
- buffer_queue Amazon SQS
- processor_fn AWS Lambda
- lake Amazon S3
- nosql_store Amazon DynamoDB
Relationships
- ingress_api invokes ingest_fn
- ingest_fn publishes notification_topic
- notification_topic publishes buffer_queue
- events_bus triggers buffer_queue
- buffer_queue invokes processor_fn
- processor_fn writes nosql_store
- processor_fn archives lake
Render it yourself.
Save the source as event-data-pipeline.arch and render it with the CLI, or ask your agent to adapt it through the ArchLex MCP server.
npx @archlex/cli render event-data-pipeline.arch --output event-data-pipeline.svgRelated examples.
- Serverless Data Lake ETL PipelineKinesis Firehose streaming to S3 Data Lake, Glue ETL transforming data, and Athena querying results
- GCP Event-Driven Data PipelineGKE publishing events to Pub/Sub, processed by Cloud Functions into BigQuery
- Modern Data Warehouse with DataflowPub/Sub streaming to Dataflow for transformation, loading into BigQuery, visualized by Looker
- Minimal Vertical SliceRDS Proxy, RDS Database, and ECS Service