JSON Field Manipulation

Purpose

JSON Field Manipulation provides a set of processors designed to modify JSON message bodies within Kafka messages flowing through Camel routes. This subtopic addresses common integration needs such as adding new fields, removing sensitive data, renaming keys, masking confidential information, restructuring payloads, and extracting specific parts of a JSON object. These transformations enable flexible message shaping, data privacy compliance, and dynamic routing decisions without altering the original source systems.

Unlike other transformation utilities that focus on routing or topic determination, this subtopic specifically targets in-message JSON content manipulation, offering fine-grained control over the message structure and contents.

Functionality

The JSON Field Manipulation processors operate on message bodies parsed as Jackson `JsonNode` trees, allowing safe and efficient traversal and modification of JSON data. Key processors include:

• InsertField: Adds a new field or element to the JSON payload. For JSON objects, it inserts a named field with the specified value. For arrays, it appends the value.

  // Inserts a field named "status" with value "processed"
  InsertField insertProcessor = new InsertField("status", "processed");
  insertProcessor.process(exchange);
  

• DropField: Removes a specified field from a JSON object, useful for stripping out sensitive or unnecessary data.

  // Removes the "password" field from JSON object
  DropField dropProcessor = new DropField();
  dropProcessor.setField("password");
  dropProcessor.process(exchange);
  

• ReplaceField: Renames fields selectively based on enabled and disabled lists, supporting bulk renaming through mappings. It filters fields to include or exclude and applies new names accordingly.

• MaskField: Masks the content of specified fields by replacing their values with nulls, default values, or custom replacements. Supports common data types and ensures type-safe masking.

• HoistField: Wraps the entire message body inside a new JSON object under a specified field name, effectively "hoisting" the existing payload one level deeper.

• ExtractField: Extracts a specific field’s value out of the JSON body either by replacing the message body or by storing it in a message header. Supports strict header existence checks and optional trimming properties.

Each processor implements either the Camel `Processor` interface or exposes a `process` method that can be invoked within Camel routes or other processors. They rely heavily on Jackson for JSON parsing and serialization, ensuring compatibility with complex JSON structures.

Relationship to Parent Topic and Other Subtopics

This subtopic is a focused specialization under the broader category of **Message Transformation Utilities**, which includes processors that route messages based on timestamps or regex patterns applied to topic headers. While the parent topic provides overall transformation support for Kafka messages, JSON Field Manipulation zeroes in on the content-level changes within JSON payloads.

By enabling direct manipulation of JSON fields, these processors complement dynamic topic routing by preparing and conditioning message data to meet downstream processing or routing requirements. For example, sensitive fields can be masked before routing to a less secure topic, or fields can be extracted into headers to influence routing decisions handled by other subtopics.

Furthermore, these JSON manipulation processors are designed to be composable and chainable within Camel routes, allowing seamless integration with other transformations, Kafka producers, and consumers.

Diagram

The following flowchart illustrates the typical processing steps when applying JSON Field Manipulation processors to a Kafka message within a Camel route:

flowchart TD
    A[Kafka Message Received] --> B[Parse JSON Body to JsonNode]
    B --> C{Processor Type?}
    C -->|InsertField| D[Add Field or Element]
    C -->|DropField| E[Remove Field]
    C -->|ReplaceField| F[Rename Fields]
    C -->|MaskField| G[Mask Field Values]
    C -->|HoistField| H[Wrap Payload under New Field]
    C -->|ExtractField| I[Extract Field to Body/Header]
    D --> J[Update JsonNode]
    E --> J
    F --> J
    G --> J
    H --> J
    I --> J
    J --> K[Serialize JsonNode to JSON]
    K --> L[Set Updated Body on Exchange]
    L --> M[Continue Camel Route Processing]

This diagram highlights the modularity and consistent processing flow regardless of the specific JSON manipulation applied.

By enabling targeted and configurable JSON field manipulations, this subtopic empowers Camel Kafka routes to enforce data shaping, privacy, and routing logic at the message content level, enriching the overall transformation capabilities within the Kafka integration framework.