Streaming Responses from an LLM in ASP.NET Core: Real-Time AI Output with Server-Sent Events

Large Language Models (LLMs) have transformed modern application development by enabling intelligent chatbots, content generators, coding assistants, and AI-powered business tools. One feature that significantly improves the user experience is response streaming, where generated text is delivered incrementally rather than waiting for the entire response to be completed.

In applications powered by AI, users expect responses to appear in real time, similar to ChatGPT. ASP.NET Core provides several mechanisms to handle streaming efficiently, allowing developers to send generated content to clients as soon as it becomes available.

What Is LLM Response Streaming?

Traditional API responses follow a request-response pattern:

1. Client sends a prompt.
2. Server waits for the LLM to generate the complete output.
3. Entire response is returned to the client.

With streaming:

1. Client sends a prompt.
2. LLM generates tokens incrementally.
3. Server forwards each token immediately to the client.
4. Users see the response appearing word by word or sentence by sentence.

This approach improves perceived performance and creates a more interactive user experience.

For developers interested in AI integration patterns, exploring technology discussions on https://mindstick.com/ can provide additional insights into modern application architectures.

Benefits of Streaming LLM Responses

Improved User Experience

Users receive instant feedback rather than waiting for the entire response.

Reduced Perceived Latency

Even if generation takes several seconds, users see progress immediately.

Better Interactivity

Streaming enables conversational interfaces that feel natural and responsive.

Efficient Resource Usage

Clients can begin processing content before generation is complete.

Streaming Options in ASP.NET Core

ASP.NET Core supports multiple approaches for real-time communication:

1. Server-Sent Events (SSE)

SSE provides a lightweight mechanism for sending continuous updates from server to browser.

Advantages:

• Simple implementation
• Native browser support
• Ideal for one-way communication
• Low overhead

2. SignalR

SignalR offers real-time bi-directional communication.

Advantages:

• Supports WebSockets
• Automatic reconnection
• Scales well for chat applications
• Supports multiple clients

3. HTTP Response Streaming

ASP.NET Core can stream data directly through the HTTP response body using asynchronous writers.

Advantages:

• Minimal complexity
• Works with existing APIs
• No additional frameworks required

Implementing Streaming with ASP.NET Core

Step 1: Create a Streaming Endpoint

The endpoint continuously writes generated tokens to the response stream.

app.MapGet("/stream", async context =>
{
    context.Response.Headers.Append(
        "Content-Type",
        "text/event-stream");

    var tokens = new[]
    {
        "Hello",
        " from",
        " ASP.NET",
        " Core",
        " streaming!"
    };

    foreach (var token in tokens)
    {
        await context.Response.WriteAsync(
            $"data: {token}\n\n");

        await context.Response.Body.FlushAsync();

        await Task.Delay(500);
    }
});

How It Works

• text/event-stream enables SSE.
• Each token is prefixed with data:.
• FlushAsync() immediately sends data to the client.
• The client receives updates without waiting for completion.

Consuming the Stream in JavaScript

The browser can listen to streamed events using the EventSource API.

const eventSource = new EventSource("/stream");

eventSource.onmessage = function (event) {
    document.getElementById("output").innerHTML +=
        event.data;
};

As tokens arrive, the UI updates instantly.

Streaming Responses from OpenAI-Compatible APIs

Most modern LLM providers support streaming output.

Example:

await foreach (var update in
    chatClient.CompleteChatStreamingAsync(messages))
{
    Console.Write(update.Text);
}

Instead of collecting the entire response, each token can be forwarded directly to the connected client.

Building an ASP.NET Core AI Streaming Service

A common architecture looks like this:

Browser
   │
   ▼
ASP.NET Core API
   │
   ▼
LLM Provider
(OpenAI/Azure OpenAI/etc.)

Workflow:

• User submits a prompt.
• ASP.NET Core calls the LLM API with streaming enabled.
• Tokens arrive progressively.
• ASP.NET Core forwards each token.
• Browser updates the interface in real time.

This architecture is commonly used in AI assistants, customer support bots, and enterprise knowledge systems.

Example Controller Implementation

[ApiController]
[Route("api/chat")]
public class ChatController : ControllerBase
{
    [HttpGet("stream")]
    public async Task Stream()
    {
        Response.ContentType =
            "text/event-stream";

        for (int i = 0; i < 10; i++)
        {
            await Response.WriteAsync(
                $"data: Token {i}\n\n");

            await Response.Body.FlushAsync();

            await Task.Delay(500);
        }
    }
}

Handling Cancellation

Users may close the page or stop generation.

Use cancellation tokens to avoid unnecessary processing.

while (!cancellationToken.IsCancellationRequested)
{
    // Generate and stream content
}

This prevents wasted API calls and reduces server load.

Error Handling During Streaming

Streaming applications should gracefully handle:

• Network interruptions
• API rate limits
• Timeout exceptions
• LLM provider failures

Example:

try
{
    await StreamResponseAsync();
}
catch (Exception ex)
{
    await Response.WriteAsync(
        $"data: Error: {ex.Message}\n\n");
}

Sending error events helps the client display meaningful messages.

Performance Considerations

Enable Asynchronous Processing

Always use async methods to avoid thread blocking.

Stream Small Chunks

Forward tokens as they arrive rather than buffering large amounts of text.

Manage Connection Limits

Long-lived streaming connections consume resources.

Implement Timeouts

Prevent abandoned connections from remaining active indefinitely.

Developers researching scalable ASP.NET Core architectures may also find useful technical discussions and implementation examples on https://answers.mindstick.com/.

When Should You Use SignalR Instead of SSE?

Choose SSE when:

• Communication is one-way.
• Only server updates are needed.
• Simplicity is important.

Choose SignalR when:

• Bi-directional communication is required.
• Multiple users participate in conversations.
• Real-time collaboration features are needed.

For most AI chat interfaces, SSE is often sufficient and easier to implement.

Security Best Practices

When streaming LLM responses:

• Validate user input.
• Authenticate API requests.
• Protect provider API keys.
• Apply rate limiting.
• Log errors securely.
• Monitor token usage.

Security becomes especially important when exposing AI capabilities to public users.

Conclusion

Streaming LLM responses in ASP.NET Core enables responsive and engaging AI-powered applications. By using technologies such as Server-Sent Events, SignalR, or direct HTTP response streaming, developers can deliver generated content to users in real time rather than forcing them to wait for complete responses.

For AI chatbots, virtual assistants, code generators, and enterprise AI solutions, streaming dramatically improves user experience and perceived performance. ASP.NET Core's asynchronous architecture makes it well-suited for implementing scalable streaming pipelines that connect users directly to modern LLM services.

As AI adoption continues to grow, mastering response streaming will become an essential skill for ASP.NET Core developers building next-generation intelligent applications.