About This Architecture
Azure Containerized Scraping and Data Engineering platform orchestrates web scraping and ETL workflows using Container Apps, Logic Apps, and PostgreSQL within a secure VNet. React frontend authenticates via Microsoft Entra ID SSO, while FastAPI microservices handle authentication, scraping, and backend processing with secrets managed through Key Vault and Managed Identity. Monthly, quarterly, and half-yearly Logic Apps trigger scheduled scraping jobs across both container services, with all images versioned through Azure Container Registry and deployed via Azure DevOps CI/CD pipelines. This architecture demonstrates enterprise-grade containerization, identity-driven security, and event-driven scheduling for data engineering teams building compliant, auditable scraping platforms on Azure.