iteration.mdx

Overview

The Iteration component is designed to facilitate iterative workflows over segmented text inputs. Its primary functionality is to split a given text input into smaller text segments based on a configurable delimiter, then execute a predefined internal workflow for each segment. This is particularly useful when the number of iterations depends dynamically on the content size or segmentation rather than a fixed loop count.

Typical use cases include:

Processing multiple paragraphs individually through a language model to avoid context confusion.
Translating lengthy documents piecewise to respect token limits.
Any scenario requiring repeated application of a component workflow over dynamically generated text segments.

Detailed Explanation

Main Component: Iteration

The Iteration component acts as a container and controller for looping workflows over text segments.

Inputs

Reference Input Variables: Link to outputs of other components or global inputs (e.g., from a Begin component).
Text Input Variables: Static text strings defined directly within the component for segmentation.

Multiple inputs can be specified, and the component supports various delimiters for splitting text:

Delimiter	Description
Comma (`,`)	Default delimiter
Line break	New line character
Tab	Tab character (`\t`)
Underline	Underscore (`_`)
Forward slash	Slash (`/`)
Dash	Hyphen (`-`)
Semicolon	Semicolon (`;`)

Functionality

Text Segmentation: The input text(s) are split into segments based on the chosen delimiter.
Workflow Iteration: For each text segment, the internal workflow is executed once.
Encapsulation: The internal workflow is encapsulated and invisible outside the Iteration component.

Internal Components

IterationItem

Role: Serves as the input node and loop manager for the internal workflow inside the Iteration component.
Visibility: Only accessible to components inside the Iteration container.
Functionality: Controls the iteration over each segmented text piece, ensuring that each segment is processed independently within the workflow.

Building Internal Workflow

Users can drag and drop other components into the Iteration component to create a custom workflow.
These internal components are hidden from the external system and rely on IterationItem for looping.
To reference the current text segment within the internal workflow, a Reference variable must be set to IterationItem in the input section of internal components.
Internal components may also access external components if needed.

Usage Example

Suppose you want to translate a long document paragraph-by-paragraph using an LLM:

Add an Iteration component.
Define the input variable as a Reference to the document text.
Set the delimiter to "Line break" to split the document into paragraphs.
Build an internal workflow containing a Generate component that performs the translation.
Inside the Generate component, reference the current paragraph via IterationItem.
Run the Iteration component to translate each paragraph sequentially.

Important Implementation Details

The iteration count is dynamic and depends on the number of segments created after splitting input text.
The IterationItem component manages the state and sequencing of each iteration internally.
Internal components added to the Iteration component become scoped only within it, ensuring modularity and encapsulation.
Referencing IterationItem inside internal components is mandatory for them to operate on the current segment.

Interaction with Other Components

Begin Component: Provides global variables that can be referenced as inputs to the Iteration component.
Generate Component (or similar): Typically nested inside the Iteration component to perform operations on each segment.
Other external components can feed data into the Iteration component via Reference variables.
Internal components can optionally reference external components as needed, but the core loop is managed internally.

Visual Diagram

flowchart TD
    Iteration["Iteration Component"]
    IterationItem["IterationItem (Internal Loop Manager)"]
    Segmentation["Text Segmentation\n(by delimiter)"]
    InternalWorkflow["Internal Workflow\n(Components)"]
    ExternalInputs["External Inputs\n(Reference / Text)"]

    ExternalInputs --> Iteration
    Iteration --> Segmentation
    Segmentation --> IterationItem
    IterationItem --> InternalWorkflow
    InternalWorkflow --> IterationItem

    style Iteration fill:#f9f,stroke:#333,stroke-width:2px
    style IterationItem fill:#bbf,stroke:#333,stroke-width:1.5px,stroke-dasharray: 5 5
    style Segmentation fill:#bfb,stroke:#333,stroke-width:1.5px
    style InternalWorkflow fill:#ffb,stroke:#333,stroke-width:1.5px
    style ExternalInputs fill:#eee,stroke:#333,stroke-width:1.5px

Summary

The Iteration component is a powerful tool to automate repetitive workflows on segmented textual data. It abstracts the complexity of dynamic loops controlled by text content size, enabling modular and scalable processing pipelines such as multi-part generation, translation, or analysis. The internal IterationItem component manages iteration state, while the encapsulated workflow ensures clean separation from the broader system.

This design fosters reusable, maintainable, and flexible workflows adaptable to various natural language processing tasks within the system.