Input Blob Artifact Saving

Purpose

Within the broader scope of the Agent Execution Runner — which orchestrates agent execution by managing message appending, agent selection, and session state updates — the subtopic Input Blob Artifact Saving addresses the need to persist binary or large data blobs included in user input messages as durable artifacts. This functionality ensures that any input blobs embedded inline in messages (such as files or binary data) are extracted and saved as separate artifacts in the session's artifact store before agent processing continues.

The existence of this feature solves several practical problems:

Prevents inline binary data from being stored directly in session event messages, which can bloat session storage and complicate message processing.
Centralizes blob data into the artifact service, enabling reuse, versioning, and efficient retrieval.
Provides a clear indication within the user message that the blob has been saved externally.
Integrates seamlessly with the session lifecycle managed by the runner.

This subtopic adds an important persistence layer for binary inputs within the agent execution flow, complementing other session and artifact management features.

Functionality

The core workflow of Input Blob Artifact Saving happens during the pre-processing of user input messages before the main agent execution starts. This is implemented inside the Runner component of the Agent Execution Runner topic, specifically in the method responsible for appending messages to the session:

func (r *Runner) appendMessageToSession(ctx agent.InvocationContext, storedSession session.Session, msg *genai.Content, saveInputBlobsAsArtifacts bool) error {
    if msg == nil {
        return nil
    }

    artifactsService := ctx.Artifacts()
    if artifactsService != nil && saveInputBlobsAsArtifacts {
        for i, part := range msg.Parts {
            if part.InlineData == nil {
                continue
            }
            fileName := fmt.Sprintf("artifact_%s_%d", ctx.InvocationID(), i)
            if _, err := artifactsService.Save(ctx, fileName, part); err != nil {
                return fmt.Errorf("failed to save artifact %s: %w", fileName, err)
            }
            // Replace the part with a text placeholder
            msg.Parts[i] = &genai.Part{
                Text: fmt.Sprintf("Uploaded file: %s. It has been saved to the artifacts", fileName),
            }
        }
    }

    event := session.NewEvent(ctx.InvocationID())
    event.Author = "user"
    event.LLMResponse = model.LLMResponse{
        Content: msg,
    }

    if err := r.sessionService.AppendEvent(ctx, storedSession, event); err != nil {
        return fmt.Errorf("failed to append event to sessionService: %w", err)
    }
    return nil
}

Key Steps in the Workflow

Detection of Inline Blobs:
The method inspects each part of the incoming user message (msg.Parts) for the presence of InlineData (blobs).
Conditional Saving:
If the feature is enabled (saveInputBlobsAsArtifacts flag) and an artifact.Service is available, each inline blob is saved as an artifact.
Artifact Naming:
Each blob is assigned a unique artifact filename based on the current invocation ID and the part index, e.g., "artifact_<invocation_id>_<part_index>".
Saving to Artifact Service:
The blob part is persisted by calling artifactsService.Save(), which stores the data in the configured artifact backend (in-memory, GCS, etc.).
Message Replacement:
After saving the blob, the original message part containing the inline blob is replaced with a textual placeholder indicating the artifact filename and that it has been saved externally.
Session Event Creation:
A new user-authored session event is created with the modified message and appended to the session via sessionService.AppendEvent.
Agent Execution Continues:
Once the blobs are safely stored and the session updated, the agent execution proceeds with the cleaned message content.

Interaction with Other Components

Session Service:
The modified user message event, now referencing saved artifacts, is persisted in the session event history.
Artifact Service:
Provides the backend storage interface for saving and later retrieving the binary blobs externalized from the message.
Agent Invocation Context:
The artifact service instance is injected into the invocation context so that downstream tools and agents can access saved artifacts during execution.
Run Configuration:
The behavior is toggled by the SaveInputBlobsAsArtifacts flag in the agent.RunConfig, allowing callers to enable or disable this feature per run.

Integration

The Input Blob Artifact Saving subtopic is tightly integrated with the core responsibilities of the Agent Execution Runner:

It enriches the message appending process by handling special cases of inline blob data.
It leverages the artifact management capabilities described in Artifact Management, by storing blobs in the session-scoped artifact store.
It ensures that session state updates contain sanitized user input messages with references to artifacts, maintaining session event consistency.
It works before the invocation of the selected agent, preserving the expected agent input format without inline binary data.
This subtopic complements the Agent Selection Logic by ensuring the input message is properly prepared regardless of which agent handles the request.

The overall flow, including this artifact saving step, fits into the agent execution lifecycle as shown below:

sequenceDiagram
participant User
participant Runner
participant ArtifactService
participant SessionService
participant Agent
User->>Runner: Send user message with inline blobs
Runner->>ArtifactService: Save each inline blob as artifact
ArtifactService-->>Runner: Artifact saved with file name
Runner->>Runner: Replace inline blob parts with placeholders
Runner->>SessionService: Append user message event with placeholders
SessionService-->>Runner: Event appended
Runner->>Agent: Run agent with cleaned message
Agent-->>Runner: Yield agent events

This sequence highlights the artifact saving step right after receiving the user message and before the agent executes.

Code Snippet Highlight

The appendMessageToSession method both detects and persists input blobs, then updates the session event with a sanitized message. This is a crucial integration point enabling the saving of input blobs as artifacts transparently:

if artifactsService != nil && saveInputBlobsAsArtifacts {
    for i, part := range msg.Parts {
        if part.InlineData == nil {
            continue
        }
        fileName := fmt.Sprintf("artifact_%s_%d", ctx.InvocationID(), i)
        if _, err := artifactsService.Save(ctx, fileName, part); err != nil {
            return fmt.Errorf("failed to save artifact %s: %w", fileName, err)
        }
        msg.Parts[i] = &genai.Part{
            Text: fmt.Sprintf("Uploaded file: %s. It has been saved to the artifacts", fileName),
        }
    }
}

This snippet encapsulates the essence of the subtopic: detecting, saving, and replacing inline blobs in the input message.

The Input Blob Artifact Saving subtopic forms a vital bridge between input message processing and artifact persistence within the agent execution flow, ensuring robust handling of binary data inputs in sessions.