About This Architecture

Kitchen Center's GCP serverless data platform orchestrates multi-source ingestion from SAP API and DB Prod using Cloud Scheduler, Workflows, and Cloud Run Jobs, landing raw data into Bronze Cloud Storage with CMEK encryption. Dataform transforms ingested Parquet and CSV into Silver and Gold BigQuery datasets with schema evolution and materialized views, while Dataplex auto-discovers assets and applies Data Lineage tagging for governance. Cloud DLP scans and masks PII before Gold layer publication, and Looker Studio consumes curated analytics—all deployed via GitOps CI/CD with Terraform IaC and Workload Identity Federation. This architecture demonstrates production-grade data governance, least-privilege security, and cost-optimized serverless compute without managing infrastructure. Fork this diagram on Diagrams.so to customize source connectors, add transformation logic, or adapt the governance framework for your own multi-tenant data lake. The modular design separates ingestion, transformation, and security concerns, making it ideal for enterprises requiring compliance, audit trails, and cross-project service account delegation.

People also ask

How do you build a secure, governed data platform on GCP using serverless services like Cloud Run and Dataform?

Kitchen Center's architecture uses Cloud Scheduler and Cloud Workflows to orchestrate Cloud Run Jobs for multi-source ingestion, Dataform for SQL-based transformation with quality checks, and Dataplex for auto-discovery and lineage tagging. Cloud DLP scans and masks PII before Gold layer publication, while CMEK encryption and Workload Identity Federation enforce least-privilege security across Bro