About This Architecture
Enterprise data platform on GCP integrating Oracle databases, email-delivered files, and Power BI sources through a multi-layer pipeline using Cloud Dataflow, Pub/Sub, Composer, and Dataproc. Data flows from ingestion through Cloud Dataflow batch jobs and event-triggered Pub/Sub topics into Cloud Composer orchestration, which coordinates Dataproc Spark transformations and Data Fusion ETL/ELT operations. Processed data lands in Cloud Storage raw zones and BigQuery central warehouse, then distributes to Travel and Operations data marts with governance via Data Catalog and security via Cloud IAM. This architecture demonstrates best practices for scalable, governed data integration with automated orchestration, enabling real-time and batch processing at enterprise scale. Fork this diagram on Diagrams.so to customize data sources, add additional marts, or adjust transformation logic for your organization's specific requirements.