When a consumer joins or leaves a group, I know a rebalance happens, but the mechanics are fuzzy. What triggers the rebalance, how does the group coordinator elect a leader, and what are the risks during this window?
Rebalance Triggers
Common triggers include explicit seekToBeginning calls, subscription changes, or broker failures. During a rebalance, the group coordinator pauses message delivery to assign partitions.
The Rebalance Process
The coordinator sends a JoinGroup request, consumers respond with their preferred assignments, and the coordinator finalizes the partition map. If a consumer is processing a record when the rebalance starts, what happens to that in-flight message?
Is there a way to minimize the impact of these pauses on latency-sensitive applications?