ClassicRebalanceListener.java

Overview

The `ClassicRebalanceListener` class is a specialized implementation of Kafka's `ConsumerRebalanceListener` interface designed for use within the Apache Camel Kafka component. Its primary responsibility is to manage partition rebalancing events in Kafka consumers by reacting to partition revocation and assignment events in a controlled and configurable manner.

This class ensures proper offset management during partition revocation when automatic offset commit is disabled, and it adapts partition assignment handling using a pluggable `PartitionAssignmentAdapter`. It plays a crucial role in maintaining Kafka consumer state consistency and smooth recovery during consumer group rebalancing events within the Apache Camel Kafka integration.

Class: ClassicRebalanceListener

public class ClassicRebalanceListener implements ConsumerRebalanceListener

Description

Implements Kafka's `ConsumerRebalanceListener` to handle partition revocation and assignment callbacks during consumer group rebalancing.

Fields

Field	Type	Description
`threadId`	`String`	Identifier of the consumer thread using this listener, used in log messages for tracing.
`configuration`	`KafkaConfiguration`	Configuration object containing Kafka consumer settings such as auto-commit state.
`assignmentAdapter`	`PartitionAssignmentAdapter`	Adapter responsible for handling partition assignment logic, facilitating resume strategies.
`commitManager`	`CommitManager`	Manages explicit offset commits when automatic commits are disabled.

Constructor

public ClassicRebalanceListener(
    String threadId, 
    KafkaConfiguration configuration,
    CommitManager commitManager, 
    Consumer<?, ?> consumer)

Parameters

threadId: Unique identifier for the consumer thread or instance.
configuration: Kafka consumer configuration instance.
commitManager: Manager responsible for committing offsets manually.
consumer: The underlying Kafka consumer instance.

Description

Initializes the listener with the necessary context and resolves a built-in resume adapter based on the configuration to handle partition assignment logic. The resolved assignment adapter is bound to the passed consumer instance.

Usage Example

KafkaConfiguration config = ...;
CommitManager commitMgr = ...;
Consumer<String, String> kafkaConsumer = ...;

ClassicRebalanceListener listener = new ClassicRebalanceListener(
    "consumer-thread-1", config, commitMgr, kafkaConsumer);

Methods

`onPartitionsRevoked`

@Override
public void onPartitionsRevoked(Collection<TopicPartition> partitions)

Description

Callback invoked by Kafka when partitions are revoked from the consumer during rebalancing. This method handles offset committing for the revoked partitions if automatic offset committing is disabled.

Parameters

partitions: Collection of TopicPartition objects that are being revoked from the consumer.

Behavior

Logs partition revocation events at debug level.
If autoCommitEnable is set to false in the configuration, explicitly commits offsets for each revoked partition using the CommitManager.

Usage Example

@Override
public void onPartitionsRevoked(Collection<TopicPartition> partitions) {
    // Automatically triggered by Kafka consumer during rebalance
}

`onPartitionsAssigned`

@Override
public void onPartitionsAssigned(Collection<TopicPartition> partitions)

Description

Callback invoked by Kafka when partitions are assigned to the consumer during rebalancing. This method delegates handling of partition assignment logic to the configured `PartitionAssignmentAdapter`.

Parameters

partitions: Collection of TopicPartition objects that have been assigned to the consumer.

Behavior

Logs partition assignment events at debug level.
Calls handlePartitionAssignment() on the assignmentAdapter to manage partition assignment-specific procedures, such as seeking offsets or resuming consumption.

Usage Example

@Override
public void onPartitionsAssigned(Collection<TopicPartition> partitions) {
    // Automatically triggered by Kafka consumer during rebalance
}

Important Implementation Details

Commit Management: The listener only commits offsets manually during partition revocation if the Kafka consumer’s configuration disables automatic commits (autoCommitEnable == false). This prevents duplicate commits or conflicts when the Kafka client manages commits automatically.
Partition Assignment Adaptation: By using a pluggable PartitionAssignmentAdapter resolved via AssignmentAdapterHelper, the listener supports different resume or assignment strategies transparently. This design provides flexibility and extensibility for different consumption patterns or recovery mechanisms.
Thread Identification: Logging includes the threadId to help trace partition events back to specific consumer threads in multi-threaded or multi-consumer scenarios.

Interaction with Other Components

Kafka Consumer (Consumer): The listener is registered with the Kafka consumer instance to receive rebalance callbacks.
KafkaConfiguration: Provides configuration settings, including whether auto-commit is enabled.
CommitManager: Handles explicit offset commits when required, ensuring offsets are safely stored before partitions are revoked.
PartitionAssignmentAdapter: Encapsulates logic for resuming or initializing partition consumption after assignment, allowing the listener to delegate detailed assignment handling.
AssignmentAdapterHelper: Utility to resolve the appropriate built-in assignment adapter based on the Kafka configuration.

Visual Diagram

classDiagram
    class ClassicRebalanceListener {
        -String threadId
        -KafkaConfiguration configuration
        -PartitionAssignmentAdapter assignmentAdapter
        -CommitManager commitManager
        +ClassicRebalanceListener(threadId, configuration, commitManager, consumer)
        +void onPartitionsRevoked(Collection~TopicPartition~ partitions)
        +void onPartitionsAssigned(Collection~TopicPartition~ partitions)
    }
    ClassicRebalanceListener ..> KafkaConfiguration : uses
    ClassicRebalanceListener ..> CommitManager : uses
    ClassicRebalanceListener ..> PartitionAssignmentAdapter : delegates to
    ClassicRebalanceListener ..> Consumer : sets consumer on adapter

Summary

The `ClassicRebalanceListener` class is a critical part of the Apache Camel Kafka consumer implementation, providing customized handling of Kafka consumer group rebalancing events. It ensures that offsets are committed properly when partitions are revoked and that partition assignment logic is delegated to a configurable adapter. This helps maintain the correctness and efficiency of message consumption in distributed Kafka consumer environments managed by Apache Camel.

End of Documentation for ClassicRebalanceListener.java