Synchronous Manual Commit

Purpose

Synchronous Manual Commit addresses the need for explicit, fine-grained control over Kafka offset commits within Apache Camel routes, enabling deterministic acknowledgment of consumed messages. Unlike automatic or asynchronous commits, this approach ensures that offset committing occurs immediately and reliably during route processing. This is critical in scenarios requiring strict processing guarantees, such as exactly-once semantics or tightly coordinated transaction boundaries where the route must confirm offset persistence before proceeding.

Functionality

This subtopic provides a factory and implementation that create manual commit objects performing **synchronous offset commits**. When a message is consumed from Kafka and processed within a Camel route, the route can invoke the manual commit to immediately commit the offset back to Kafka or an offset repository. This commit blocks until Kafka acknowledges the offset commit, ensuring the offset state is durable before route execution continues.

Key workflow steps include:

1. Manual Commit Object Creation:
   The DefaultKafkaManualCommitFactory instantiates a DefaultKafkaManualSyncCommit object for each consumed record. This object encapsulates the Kafka record details, Camel exchange data, and the commit manager responsible for committing offsets.

2. Synchronous Commit Invocation:
   Within the route, the commit() method on the manual commit object is called explicitly. This triggers the commit manager to synchronously commit offsets, blocking the caller until the commit completes.

3. Commit Manager Interaction:
   The commit manager handles the actual communication with Kafka or the offset store, abstracting commit strategies. In the synchronous case, it performs a blocking commit call to ensure offset persistence.

4. Error Handling:
   If commit fails, exceptions can be propagated immediately, allowing the route to decide on retry or error handling strategies.

This approach contrasts with asynchronous manual commits, which return immediately and perform offset commits in the background, trading immediacy for throughput.

Code Snippet Illustrating Factory Usage

public class DefaultKafkaManualCommitFactory implements KafkaManualCommitFactory {

    @Override
    public KafkaManualCommit newInstance(
            CamelExchangePayload camelExchangePayload, 
            KafkaRecordPayload kafkaRecordPayload, 
            CommitManager commitManager) {
        return new DefaultKafkaManualSyncCommit(camelExchangePayload, kafkaRecordPayload, commitManager);
    }
}

This factory method creates synchronous manual commit instances wired with the current exchange, record, and commit manager.

Integration and Relationship

Synchronous Manual Commit integrates tightly with:

• Parent Manual Commit Support:
  It is a concrete implementation of the manual commit API, complementing asynchronous manual commit implementations by offering a blocking commit variant.

• Commit Management:
  Works with the commit manager abstraction to execute offset commits. The synchronous commit relies on commit managers that support blocking commit calls.

• Kafka Consumer:
  Manual commit instances are attached to Camel exchanges during Kafka message consumption, enabling routes to selectively commit offsets after successful processing.

• Error Handling Strategies:
  Because commits are synchronous, errors during commit are surfaced immediately, allowing integration with error handling policies defined in the route.

This subtopic extends the manual commit support by providing a reliable and immediate offset commit mechanism, essential for use cases requiring strict processing guarantees and fine control over offset acknowledgment timing.

Diagram

sequenceDiagram
    participant Route as Camel Route
    participant ManualCommit as DefaultKafkaManualSyncCommit
    participant CommitMgr as CommitManager
    participant Kafka as Kafka Broker

    Route->>ManualCommit: invoke commit()
    ManualCommit->>CommitMgr: commit offset synchronously
    CommitMgr->>Kafka: commit offsets (blocking)
    Kafka-->>CommitMgr: acknowledge commit
    CommitMgr-->>ManualCommit: commit success
    ManualCommit-->>Route: commit complete

This sequence diagram illustrates the synchronous manual commit flow, showing the blocking offset commit from the route through the manual commit object and commit manager to Kafka, and back.

Synchronous Manual Commit provides deterministic, blocking offset commit capabilities within Camel routes, ensuring immediate acknowledgment of consumed Kafka messages and supporting robust integration scenarios requiring tight offset control.