---
title: "Implement Kafka in .NET Core API"  
description: "Apache Kafka has become one of the most popular distributed event streaming platforms for building scalable and high-performance applications. In modern microse"  
author: "ICSM Computer"  
published: 2026-05-07  
updated: 2026-05-07  
canonical: https://answers.mindstick.com/blog/272/implement-kafka-in-dot-net-core-api  
category: "software"  
tags: ["software", "software development"]  
reading_time: 5 minutes  

---

# Implement Kafka in .NET Core API

Apache Kafka has become one of the most popular distributed event streaming platforms for building scalable and high-performance applications. In modern microservice architectures, Kafka enables asynchronous communication between services, improves reliability, and handles millions of events efficiently.

## What is Kafka?

Apache Kafka is an open-source distributed event streaming platform used for:

- Real-time data streaming
- Event-driven architecture
- Message brokering
- Log aggregation
- [Data pipelines](https://www.mindstick.com/articles/337444/what-are-data-pipelines-and-their-importance-in-artificial-intelligence)

Kafka works using three major components:

- **Producer** → Sends messages
- **Consumer** → Receives messages
- **Broker** → Kafka server managing topics and messages

![Implement Kafka in .NET Core API](https://answers.mindstick.com/blogs/13da7cda-5549-4d5c-985c-b7ca0f539738/images/0707240a-9eb0-41c3-a8d3-7cc2dc46b2c1.png)

## Why Use Kafka with .NET Core?

Using Kafka in .NET applications provides several benefits:

- High throughput
- Fault tolerance
- Scalability
- Loose coupling between services
- Asynchronous processing
- Real-time event streaming

Typical use cases include:

- Order processing systems
- Notification services
- Payment systems
- Audit logging
- Microservices communication

## Prerequisites

Before starting, make sure you have:

- .NET 8 SDK installed
- Docker installed
- Kafka running locally
- Basic knowledge of ASP.NET Core Web API

## Step 1: Run Kafka Using Docker

Create a `docker-compose.yml` file:

```plaintext
version: '3'

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:latest
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181

  kafka:
    image: confluentinc/cp-kafka:latest
    ports:
      - "9092:9092"

    environment:
      KAFKA_BROKER_ID: 1
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181

      KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://localhost:9092
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
```

Run the containers:

```plaintext
docker-compose up -d
```

This will start:

- Zookeeper
- Kafka Broker

## Step 2: Create ASP.NET Core Web API

Create a new project:

```plaintext
dotnet new webapi -n KafkaDemoAPI
```

Navigate to the project:

```plaintext
cd KafkaDemoAPI
```

## Step 3: Install Kafka NuGet Package

Install the Kafka client package:

```plaintext
dotnet add package Confluent.Kafka
```

The official Kafka .NET client is maintained by Confluent.

Package documentation:

[Confluent.Kafka NuGet Package](https://www.nuget.org/packages/Confluent.Kafka/)

## Step 4: Create Kafka Producer Service

Create a folder named `Services`.

## KafkaProducerService.cs

```cs
using Confluent.Kafka;
using System.Text.Json;

namespace KafkaDemoAPI.Services
{
    public class KafkaProducerService
    {
        private readonly ProducerConfig _config;

        public KafkaProducerService()
        {
            _config = new ProducerConfig
            {
                BootstrapServers = "localhost:9092"
            };
        }

        public async Task ProduceMessageAsync<T>(string topic, T message)
        {
            using var producer =
                new ProducerBuilder<Null, string>(_config).Build();

            var jsonMessage =
                JsonSerializer.Serialize(message);

            var result = await producer.ProduceAsync(
                topic,
                new Message<Null, string>
                {
                    Value = jsonMessage
                });

            Console.WriteLine(
                $"Message sent: {result.Value}");
        }
    }
}
```

## Step 5: Register Service in Program.cs

```cs
using KafkaDemoAPI.Services;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddSingleton<KafkaProducerService>();

builder.Services.AddControllers();

var app = builder.Build();

app.MapControllers();

app.Run();
```

## Step 6: Create API Controller

Create `Controllers/OrderController.cs`

```cs
using KafkaDemoAPI.Services;
using Microsoft.AspNetCore.Mvc;

namespace KafkaDemoAPI.Controllers
{
    [ApiController]
    [Route("api/[controller]")]
    public class OrderController : ControllerBase
    {
        private readonly KafkaProducerService _producer;

        public OrderController(
            KafkaProducerService producer)
        {
            _producer = producer;
        }

        [HttpPost]
        public async Task<IActionResult> CreateOrder(
            [FromBody] object order)
        {
            await _producer.ProduceMessageAsync(
                "order-topic",
                order);

            return Ok(new
            {
                Message = "Order published to Kafka"
            });
        }
    }
}
```

## Step 7: Create Kafka Consumer

Now create a background consumer service.

## KafkaConsumerService.cs

```cs
using Confluent.Kafka;

namespace KafkaDemoAPI.Services
{
    public class KafkaConsumerService : BackgroundService
    {
        private readonly ConsumerConfig _config;

        public KafkaConsumerService()
        {
            _config = new ConsumerConfig
            {
                BootstrapServers = "localhost:9092",
                GroupId = "order-group",
                AutoOffsetReset = AutoOffsetReset.Earliest
            };
        }

        protected override Task ExecuteAsync(
            CancellationToken stoppingToken)
        {
            return Task.Run(() =>
            {
                using var consumer =
                    new ConsumerBuilder<Ignore, string>(_config)
                    .Build();

                consumer.Subscribe("order-topic");

                while (!stoppingToken.IsCancellationRequested)
                {
                    var consumeResult =
                        consumer.Consume(stoppingToken);

                    Console.WriteLine(
                        $"Received Message: {consumeResult.Message.Value}");
                }

            }, stoppingToken);
        }
    }
}
```

## Step 8: Register Consumer Service

Update `Program.cs`

```cs
builder.Services.AddHostedService<KafkaConsumerService>();
```

## Step 9: Run the Application

Start the API:

```plaintext
dotnet run
```

Send a POST request:

```plaintext
POST /api/order
```

Request Body:

```plaintext
{
  "orderId": 101,
  "product": "Laptop",
  "price": 75000
}
```

You will see the message consumed in the console.

## Kafka Message Flow

The workflow looks like this:

```plaintext
Client Request
      ↓
.NET Core API
      ↓
Kafka Producer
      ↓
Kafka Topic
      ↓
Kafka Consumer
      ↓
Background Processing
```

## Best Practices for Kafka in .NET

## 1. Use Strongly Typed Models

Instead of `object`, use DTOs.

```cs
public class OrderModel
{
    public int OrderId { get; set; }
    public string Product { get; set; }
    public decimal Price { get; set; }
}
```

## 2. Store Kafka Configurations in appsettings.json

```plaintext
"Kafka": {
  "BootstrapServers": "localhost:9092",
  "Topic": "order-topic",
  "GroupId": "order-group"
}
```

## 3. Implement Retry Policies

Use retry mechanisms for transient failures.

## 4. Enable Logging and Monitoring

Monitor:

- consumer lag
- failed messages
- broker health
- throughput

Useful tools:

- [Kafka UI](https://github.com/provectus/kafka-ui)
- [Conduktor](https://www.conduktor.io/)

## Advantages of Kafka Architecture

| Feature | Benefit |
| --- | --- |
| Scalability | Handles huge traffic |
| Durability | Messages persisted safely |
| High Throughput | Millions of messages |
| Fault Tolerance | Distributed brokers |
| Loose Coupling | Better microservice architecture |

## Common Real-World Kafka Use Cases

Companies use Kafka for:

- Order processing
- Banking transactions
- Real-time analytics
- Fraud detection
- Notification systems
- Event sourcing
- Activity tracking

Many large-scale platforms including [Netflix](https://www.netflix.com/), [Uber](https://www.uber.com/), and [LinkedIn](https://www.linkedin.com/) use Kafka extensively.

## Conclusion

Implementing Kafka in a .NET Core API is straightforward using the `Confluent.Kafka` package. Kafka enables highly scalable and asynchronous communication between services, making it ideal for modern distributed systems and microservices.

By integrating Kafka with .NET Core, you can build resilient and event-driven applications capable of handling enterprise-scale workloads efficiently.

## Helpful Resources

- [Apache Kafka Official Documentation](https://kafka.apache.org/documentation/?utm_source=chatgpt.com)
- [ASP.NET Core Documentation](https://learn.microsoft.com/aspnet/core/?utm_source=chatgpt.com)
- [Confluent Kafka .NET Docs](https://docs.confluent.io/kafka-clients/dotnet/current/overview.html?utm_source=chatgpt.com)

---

Original Source: https://answers.mindstick.com/blog/272/implement-kafka-in-dot-net-core-api

Copyright © MindStick Software Pvt. Ltd. This Markdown version is provided for developers, AI systems, and offline reading.