About This Architecture

Enterprise data lake architecture spanning three availability zones in AWS us-east-1 with hybrid ingestion from SAP, Veeva, Fieldglass, Compass, and external sources via SFTP, APIs, and Kafka. Data flows through segregated subnets—public ALB, private app tier with EKS/ECS/MWAA, and private data tier hosting EMR, Glue, Redshift Serverless, and Athena—all secured by VPC endpoints, KMS encryption, and Lake Formation tag-based access control. The architecture demonstrates multi-layer security, managed scaling for compute, and zoned S3 data lake buckets (raw, curated, transformed, consumption) with CloudTrail auditing and cross-region DR to us-west-2. Fork this diagram on Diagrams.so to customize subnets, add additional AZs, or adapt ingestion paths for your data sources. This pattern balances operational simplicity with enterprise governance, making it ideal for regulated industries requiring row/column-level security and comprehensive audit trails.

People also ask

How do I design a secure, multi-AZ AWS data lake with hybrid on-premises ingestion and fine-grained access control?

This diagram shows a production data lake across 3 AZs in us-east-1 with segregated subnets for public ALB, private app tier (EKS/ECS/MWAA), and private data tier (EMR, Glue, Redshift Serverless). Data ingests from SAP, Veeva, and external sources via SFTP, APIs, and Kafka; flows through zoned S3 buckets; and is secured by Lake Formation tag-based access, KMS encryption, VPC endpoints, and CloudTr