About This Architecture
Real-time IoT data lake pipeline ingesting sensor telemetry via AWS IoT Core MQTT broker into Kinesis Data Streams, with parallel anomaly detection and stream transformation. Raw JSON data (~5000 msg/s) lands in S3 Raw Tier, then flows through Glue ETL batch jobs to Curated Tier (hourly Parquet), and finally to Aggregated Tier (daily Parquet) for historical analysis. Redshift and Athena serve curated and aggregated data to QuickSight dashboards, enabling real-time monitoring and ad-hoc exploration of IoT metrics. This architecture demonstrates the medallion lakehouse pattern with hot-path anomaly detection and cold-path batch enrichment. Fork this diagram to customize ingestion rates, add additional analytics services, or adapt for your IoT use case.