How does Kafka's consumer group protocol manage rebalancing?

Asked 14 hours ago 30 views

0

When a consumer joins or leaves a group, I know a rebalance happens, but the mechanics are fuzzy. What triggers the rebalance, how does the group coordinator elect a leader, and what are the risks during this window?

Rebalance Triggers

Common triggers include explicit seekToBeginning calls, subscription changes, or broker failures. During a rebalance, the group coordinator pauses message delivery to assign partitions.

The Rebalance Process

The coordinator sends a JoinGroup request, consumers respond with their preferred assignments, and the coordinator finalizes the partition map. If a consumer is processing a record when the rebalance starts, what happens to that in-flight message?

Is there a way to minimize the impact of these pauses on latency-sensitive applications?

0 Answers


Write Your Answer