About This Architecture

Cross-account data lake architecture connecting AWS Glue ETL jobs in Datalake PRD account to legacy SQL Server databases in LAIVE account via AWS Transit Gateway. Data engineers and analytics users trigger Glue jobs that read from S3 Datalake Storage and write to SELLOUTBD and SATEL SQL Server instances across VPCs using private subnets and TGW attachments. This pattern eliminates internet exposure, enforces least-privilege IAM roles, and maintains network isolation while enabling seamless ETL connectivity to on-premises-style databases. Fork this diagram on Diagrams.so to customize subnets, add VPC endpoints, or integrate additional SQL Server targets. The architecture demonstrates hybrid cloud best practices: private connectivity, cross-account access control, and centralized monitoring via CloudWatch.

People also ask

How do I securely connect AWS Glue ETL jobs to SQL Server databases in a different AWS account?

Use AWS Transit Gateway to establish private connectivity between your Datalake PRD account (hosting Glue jobs and S3 storage) and LAIVE account (hosting SQL Server EC2 instances). Deploy TGW attachments in private subnets, configure IAM roles for least-privilege access, and monitor data flow via CloudWatch. This eliminates internet exposure while maintaining cross-account isolation.