About This Architecture

Enterprise medallion architecture on AWS ingests data from PostgreSQL, SQL Server, MongoDB, SAP, Salesforce, and external files using DMS, Kinesis Data Streams, Glue ETL, Lambda, and AppFlow. Raw data lands in S3 Bronze layer, flows through Glue Crawler and EMR Spark processing to Silver (curated) and Gold (aggregated) layers, all cataloged by Glue Data Catalog and governed by Lake Formation. Consumption layer serves Power BI, QuickSight, SageMaker ML models, Bedrock Gen AI, and Athena ad-hoc queries via API Gateway, with Redshift data marts for structured analytics. Comprehensive governance enforced through IAM, KMS encryption, CloudTrail audit logs, CloudWatch monitoring, Macie data discovery, GuardDuty threat detection, and Config compliance tracking. Fork this diagram on Diagrams.so to customize ingestion sources, add transformation logic, or adapt the medallion layers for your data platform requirements.

People also ask

How do I design an AWS data lake with medallion architecture and Lake Formation governance?

This AWS medallion architecture ingests from PostgreSQL, MongoDB, SAP, Salesforce via DMS, Kinesis, Glue, AppFlow into S3 Bronze layer, transforms through Glue ETL and EMR Spark to Silver and Gold layers, governed by Lake Formation, serving Redshift, QuickSight, SageMaker, and Bedrock with IAM, KMS, Macie security.