--- title: "What is the recommended architecture for building an AI chatbot using .NET?" description: "What is the recommended architecture for building an AI chatbot using .NET?" author: "Manish Kumar" published: 2026-08-06 updated: 2026-08-10 canonical: https://answers.mindstick.com/qa/117026/what-is-the-recommended-architecture-for-building-an-ai-chatbot-using-dot-net category: "asp.net" tags: ["asp.net", "c#", ".net programming"] reading_time: 7 minutes --- # What is the recommended architecture for building an AI chatbot using .NET? ## What is the recommended architecture for building an AI chatbot using .NET? ## Answers ### Answer by Manish Kumar For a **production AI chatbot in .NET**, I’d recommend a **modular monolith / Clean Architecture first**, with an AI orchestration layer that can evolve into an agent architecture later. You generally don't need microservices just because the application uses an LLM. Microsoft's current .NET AI guidance supports multiple model providers through `Microsoft.Extensions.AI`, while Semantic Kernel and the newer agent/tooling ecosystem provide orchestration capabilities. ### Recommended architecture ```plaintext ┌──────────────────────┐ │ Web / Mobile UI │ │ Blazor / React / etc │ └──────────┬───────────┘ │ HTTPS / SignalR ▼ ┌──────────────────────┐ │ ASP.NET Core API │ │ Auth / Rate limiting │ │ Streaming / Sessions │ └──────────┬───────────┘ │ ┌──────────▼───────────┐ │ Application Layer │ │ Chat orchestration │ │ Conversation logic │ └──────────┬───────────┘ │ ┌────────────────┼─────────────────┐ ▼ ▼ ▼ ┌─────────────┐ ┌──────────────┐ ┌──────────────┐ │ LLM Gateway │ │ RAG Service │ │ Tool/Agent │ │ │ │ │ │ Orchestrator │ └──────┬──────┘ └──────┬───────┘ └──────┬───────┘ │ │ │ ▼ ▼ ▼ Azure OpenAI / Azure AI Search APIs / DB / OpenAI / other Vector + Hybrid Business funcs │ └──────────────┬─────────────────┘ ▼ ┌──────────────────┐ │ Conversation DB │ │ Redis / SQL etc. │ └──────────────────┘ ``` ### 1. ASP.NET Core as the application boundary Use **ASP.NET Core** for the API rather than allowing your frontend to communicate directly with the LLM. Responsibilities: - Authentication/authorization - Conversation/session management - Rate limiting - Input validation - Streaming responses - Tenant/user isolation - Audit logging - Calling the application layer Microsoft's .NET architecture guidance recommends separating application concerns even when deploying them as a single application, and Clean Architecture is particularly appropriate for non-trivial ASP.NET Core applications. ![What is the recommended architecture for building an AI chatbot using .NET?](https://answers.mindstick.com/questionanswer/6c114049-5ee0-4f5b-927b-96c0d353c6b0/images/34bb5ada-0687-49f1-b3bc-b23fc8935998.jpg) A practical project structure: ```plaintext src/ Chatbot.Api/ Controllers/ Hubs/ Middleware/ Chatbot.Application/ Chat/ Agents/ RAG/ Tools/ Interfaces/ Chatbot.Domain/ Conversations/ Users/ Documents/ Chatbot.Infrastructure/ AI/ Search/ Persistence/ ExternalServices/ tests/ Chatbot.UnitTests/ Chatbot.IntegrationTests/ ``` ### 2. Put an LLM abstraction behind your application Don't scatter calls to `OpenAIClient` or `AzureOpenAI` throughout your controllers. Instead: ```cs public interface IChatService { IAsyncEnumerable StreamAsync( ChatRequest request, CancellationToken cancellationToken); } ``` Then your infrastructure can implement it using the model/provider you choose. This is particularly useful because the current .NET AI ecosystem supports providers including OpenAI, Azure OpenAI, Azure AI Foundry, Ollama, Gemini and others through common .NET abstractions. For a Microsoft/Azure enterprise application, I'd typically start with: ```plaintext ASP.NET Core ↓ Microsoft.Extensions.AI / orchestration layer ↓ Azure OpenAI / Azure AI Foundry ``` ### 3. Add RAG when the chatbot needs your data If the bot needs to answer questions about: - company documents - manuals - policies - product information - SharePoint content - internal knowledge bases use **RAG (Retrieval-Augmented Generation)** rather than putting all of that information into the prompt. ```plaintext User question ↓ Query understanding ↓ Search ┌───┴──────────────┐ │ │ Keyword Vector search search │ │ └───────┬──────────┘ ▼ Semantic ranking ↓ Relevant chunks ↓ LLM ↓ Answer + citations ``` Azure AI Search supports keyword, vector and hybrid search, as well as semantic ranking. ([Microsoft Learn](https://learn.microsoft.com/en-us/dotnet/api/overview/azure/search.documents-readme?view=azure-dotnet&utm_source=chatgpt.com)) For an Azure-centric implementation, a very good starting stack is: > ## Blob Storage → ingestion/chunking → Azure AI Search → Azure OpenAI → ASP.NET Core Microsoft also documents a .NET RAG architecture combining Azure OpenAI, Azure AI Search and storage, including automatic citations. ![What is the recommended architecture for building an AI chatbot using .NET?](https://answers.mindstick.com/questionanswer/6c114049-5ee0-4f5b-927b-96c0d353c6b0/images/fd6fd893-62a1-485f-8943-b71b3e63ce23.jpg) ### 4. Treat tools as separate capabilities If the chatbot needs to **do things**, don't give the LLM direct access to your database. Expose controlled tools: ```cs public interface IOrderTools { Task GetOrderStatusAsync(string orderId); Task CancelOrderAsync(string orderId); } ``` The architecture becomes: ```plaintext LLM │ decides to call │ ▼ Tool interface │ ▼ Application service │ ▼ Authorization + validation │ ▼ External system ``` This gives you a crucial security boundary. Microsoft's current agent architecture similarly separates [**agents, skills and tools**](https://learn.microsoft.com/en-us/agents/architecture/search-tool-use-architectures), with tools representing discrete operations exposed through defined interfaces/schemas. ### 5. Use Semantic Kernel selectively **Semantic Kernel** is useful when your chatbot needs orchestration, plugins/tools, memory, structured AI workflows, or agent-like behavior. Its [kernel](https://learn.microsoft.com/en-us/semantic-kernel/concepts/kernel) acts as a central dependency-injection/orchestration point for AI services and plugins. For example: ```plaintext ChatService │ └── Semantic Kernel ├── Chat completion ├── RAG plugin ├── Order plugin ├── Customer plugin └── Notification plugin ``` I wouldn't introduce Semantic Kernel merely because "it's an AI application." For a simple chatbot: ```plaintext ASP.NET Core ↓ Microsoft.Extensions.AI ↓ LLM ``` may be enough. Introduce an orchestration/agent framework once you actually have multiple tools, workflows, or agents. ### 6. Keep conversation state outside the LLM Don't rely on the model to remember everything. Use: ## SQL/PostgreSQL/SQL Server for durable conversation records: ```plaintext Conversation ├── Id ├── UserId ├── CreatedAt └── Title Message ├── Id ├── ConversationId ├── Role ├── Content ├── Tokens └── CreatedAt ``` And optionally **Redis** for: - short-lived state - caching - rate limiting - distributed locks - frequently accessed data For long conversations, don't blindly send the entire history to the model. Use: ```plaintext Recent messages + Conversation summary + Relevant retrieved memories/documents ↓ LLM ``` ### 7. Security should be designed into the AI layer This is especially important for enterprise chatbots. Your pipeline should look more like: ```plaintext User ↓ Authentication ↓ Authorization ↓ Prompt/input validation ↓ Conversation context ↓ RAG with permission filtering ↓ LLM ↓ Tool authorization ↓ Output validation ↓ User ``` For example, if Alice isn't allowed to access document X, **the RAG layer must prevent document X from reaching the model**. Don't rely on the LLM to decide whether Alice is allowed to see it. For tools that modify data, add explicit authorization and preferably confirmation for consequential operations. ![What is the recommended architecture for building an AI chatbot using .NET?](https://answers.mindstick.com/questionanswer/6c114049-5ee0-4f5b-927b-96c0d353c6b0/images/6741d981-1414-47c6-befc-86509ab2db9f.jpg) ### 8. Observability is part of the architecture Track at least: ```plaintext Request ├── latency ├── model ├── input/output tokens ├── estimated cost ├── retrieved documents ├── tool calls ├── errors └── user feedback ``` You want to be able to answer: > "Why did the chatbot give this answer?" That means retaining enough trace information to reconstruct: ```plaintext User question ↓ Retrieved context ↓ Prompt/model configuration ↓ Tool calls ↓ Model response ``` Also build an evaluation set of representative questions and measure things such as answer correctness, retrieval quality, citation accuracy, latency and cost. ## What I'd actually deploy For a typical enterprise chatbot, my starting architecture would be: | Layer | Recommendation | | --- | --- | | UI | React / Blazor | | API | ASP.NET Core | | Architecture | Clean Architecture / modular monolith | | AI abstraction | `Microsoft.Extensions.AI` | | Orchestration | Semantic Kernel or Agent Framework when needed | | Model | Azure OpenAI / Azure AI Foundry | | RAG | Azure AI Search | | Primary DB | SQL Server/PostgreSQL | | Cache | Redis | | Documents | Azure Blob Storage | | Authentication | Microsoft Entra ID | | Secrets | Managed Identity + Key Vault | | Observability | OpenTelemetry + Application Insights | | Deployment | Azure App Service / Container Apps | | Async jobs | Azure Service Bus / background workers | Microsoft's current .NET AI documentation specifically covers [chat applications, RAG, vector search, agents](https://learn.microsoft.com/en-us/dotnet/ai/), tool execution and evaluation, so this stack aligns well with the current .NET ecosystem. ### The important architectural principle ## Don't start with "microservices + agents + vector database." Start with: ```plaintext ASP.NET Core ↓ Application/Chat service ↓ LLM ↓ SQL ``` Then add capabilities as requirements appear: ```plaintext Need company knowledge? → RAG / AI Search Need actions? → Tools Need complex orchestration? → Agent framework / Semantic Kernel Need scale? → Redis + queues + horizontal scaling Need multiple services? → Extract services only when justified ``` That gives you a system that's straightforward to develop and test initially, while leaving room for a sophisticated agent/RAG architecture later. Microsoft's ASP.NET Core architecture guidance likewise notes that a single deployed application is often easier to build, deploy and debug than prematurely splitting it into services. If you're building **a specific chatbot** (e.g. customer support, internal enterprise knowledge bot, SQL/database assistant, or an agent that performs actions), the optimal architecture changes substantially. --- Original Source: https://answers.mindstick.com/qa/117026/what-is-the-recommended-architecture-for-building-an-ai-chatbot-using-dot-net Copyright © MindStick Software Pvt. Ltd. This Markdown version is provided for developers, AI systems, and offline reading.