0
My team is debating whether to build stateful stream processing with a hand-rolled consumer or adopt Kafka Streams. I understand Kafka Streams is a client library, but I'm unclear on the specific operational and development advantages it provides over writing everything from scratch.
State Management and Fault Tolerance
One thing that keeps coming up is state stores. When you need to perform aggregations or windowed joins, how does Kafka Streams handle local state versus changelog topics? Is the abstraction worth the overhead?
Operational Simplicity
With a custom consumer, you're responsible for offset management, exactly-once semantics, and rebalancing logic. Kafka Streams abstracts a lot of this, but at what cost in terms of debugging visibility and tuning flexibility?
