About This Architecture

ANI is a multi-tenant, enterprise-grade generative AI search platform built on Azure with a sophisticated layered architecture spanning clients, edge security, identity federation, microservices, LLM orchestration, and 25+ data connectors. Clients access the platform via web browser, admin dashboard, Slack/Discord bots, or API; requests flow through Azure Front Door with WAF and DDoS protection, then authenticate via Keycloak and Azure Entra ID before reaching Next.js 15 frontend and FastAPI backend services. The platform unifies multiple LLM providers (OpenAI, Azure OpenAI, Gemini, Claude, Ollama) through LiteLLM Gateway with token management and fallback routing, while Celery workers asynchronously index documents from SharePoint, Google Drive, GitHub, Jira, Slack, email, and REST APIs into PostgreSQL with pgvector embeddings and Neo4j knowledge graphs. This architecture demonstrates enterprise best practices for multi-tenancy, security, observability, and AI integration at scale. Fork and customize this diagram on Diagrams.so to adapt the layered design for your own AI search or RAG platform needs.

People also ask

How do you build a multi-tenant enterprise AI search platform on Azure with support for multiple LLM providers and 25+ data sources?

ANI demonstrates a layered architecture using Azure Front Door for edge security, Keycloak and Entra ID for multi-tenant identity, Next.js 15 and FastAPI microservices for application logic, LiteLLM Gateway for unified LLM orchestration across OpenAI, Azure OpenAI, Gemini, Claude, and Ollama, and Celery workers for asynchronous indexing from SharePoint, Google Drive, GitHub, Jira, Slack, and other