About This Architecture
Kubernetes data processing cluster integrating cert-manager for automated TLS certificate lifecycle, Vault for secrets management, and external-secrets-operator for credential synchronization across namespaces. Data flows from ingress through the data-api service to etl-worker pods, spark-cluster statefulsets, and batch jobs, with all traffic encrypted via Let's Encrypt ACME-issued certificates. Network policies, RBAC, and service accounts enforce least-privilege access, while HPA scales etl-workers based on demand and persistent volumes provide durable storage for ETL and Spark workloads. Fork this diagram to customize your own multi-tenant data platform, adjust node counts, or swap Vault for AWS Secrets Manager.