Instruction Template Processing

Overview

Instruction Template Processing is a core mechanism that dynamically injects session state and artifact data into large language model (LLM) prompts at runtime. This capability enables agents to generate contextually rich instructions by substituting placeholders in templates with the current session’s state values or the contents of stored artifacts. The module addresses the problem of making LLM instructions adaptive to evolving session information and external data, thus enhancing the relevance and precision of AI agent interactions.

By processing instruction templates with dynamic data injection, the system decouples instruction authoring from runtime data availability, allowing template reuse and flexible prompt customization without manual intervention.

Core Concepts

Instruction Templates

Instruction templates are textual prompts or instructions containing placeholders enclosed in curly braces {}. These placeholders represent variables or artifacts that should be replaced by actual values from the session state or artifact storage at invocation time.

Example Template:

Hello {user_name}, your current task status is {app:task_status}.
Please refer to the document: {artifact.project_overview}.

Here, {user_name} and {app:task_status} are session state variables, while {artifact.project_overview} refers to the contents of an artifact named project_overview.

Placeholder Syntax and Semantics

Session State Variables: {key_name}, where key_name follows identifier rules (letters, digits, underscores). Names can be optionally prefixed with namespaces such as app:, user:, or temp: (e.g., {app:task_status}).
Artifacts: {artifact.file_name}, where file_name is the artifact's identifier.
Optional Placeholders: Adding ? suffix (e.g., {user_email?}, {artifact.readme?}) signals that missing data should not cause errors but be replaced with an empty string.
Invalid placeholders are left unmodified in the output.

Functionality and Workflow

At a high level, the Instruction Template Processing module performs the following:

Parsing: Scans instruction text for placeholders matching the pattern {...}.
Resolution: For each placeholder:
- Determines whether it refers to a session state variable or an artifact.
- Validates the variable name or artifact identifier.
- Retrieves the corresponding value from the session state or artifact service.
- Applies optional logic for missing data.
Replacement: Substitutes placeholders with their resolved string values.
Integration: Appends the processed instruction text to the outgoing LLM request payload.

This workflow is triggered automatically during LLM request construction, ensuring that instructions reflect the latest session and artifact data.

Interaction with System Components

Invocation Context (agent.InvocationContext): Provides access to the current session and artifact services. The module operates within this context to fetch required data.
Session Service: Supplies session state key-value pairs that populate placeholders referencing state variables.
Artifact Service: Enables loading of artifacts’ textual content, injected into prompts where artifact placeholders appear.
LLM Request (model.LLMRequest): The processed instruction text is appended to the LLM request’s instructions, influencing prompt generation.
Agent State (internal/llminternal.State): Holds the instruction templates or providers that the module processes.

The module is tightly coupled with the agent’s invocation lifecycle: when preparing an LLM request, it enriches instructions by injecting dynamic data, as seen in the instructionsRequestProcessor function which combines global and agent-specific instructions.

Key Files and Their Roles

`internal/llminternal/instruction_processor.go`

This file contains the main implementation. Its responsibilities include:

Template Processing: The InjectSessionState function processes the input template string, identifying placeholders using regex and replacing them with session or artifact values.
Placeholder Replacement Logic: Implemented in replaceMatch, which handles:
- Artifact loading via the artifact service.
- Session state retrieval via the session state interface.
- Optional placeholder handling.
- Validation of placeholder names (isValidStateName and isIdentifier).
Instruction Appending: Functions appendInstructions and appendGlobalInstructions inject processed instructions into the LLM request.
Integration: instructionsRequestProcessor hooks into the agent invocation pipeline, ensuring instructions are processed and appended before the LLM call.

Example snippet illustrating template processing:

func InjectSessionState(ctx agent.InvocationContext, template string) (string, error) {
	// Find all {placeholders} and replace with session/artifact values.
	for each placeholder in template {
		replacement, err := replaceMatch(ctx, placeholder)
		if err != nil {
			return "", err
		}
		replace placeholder with replacement
	}
	return processedTemplate, nil
}

`util/instructionutil/instruction.go`

Provides a public-facing wrapper around the internal processing function, ensuring that only a read-only context is used when injecting session state into instruction templates. This enforces immutability guarantees during template injection.

`internal/llminternal/instruction_processor_test.go`

Contains comprehensive unit tests for instruction template processing, verifying scenarios including:

Successful injection of session state values.
Artifact content injection.
Handling of optional placeholders.
Error conditions on missing required variables or artifacts.
Validation of placeholder syntax and prefix rules.

Instruction Template Processing Flow

flowchart TD
Start[Start: Receive Instruction Template]
Parse[Parse Template for Placeholders]
ForEach[For Each Placeholder]
IsArtifact{Is Placeholder an Artifact?}
LoadArtifact[Load Artifact Content]
GetState[Get Session State Value]
IsOptional{Is Placeholder Optional?}
Replace[Replace Placeholder with Value]
Error{Error?}
SkipOrFail[Skip or Fail Based on Optional Flag]
End[Return Processed Instruction String]
Start --> Parse --> ForEach
ForEach --> IsArtifact
IsArtifact -->|Yes| LoadArtifact
IsArtifact -->|No| GetState
LoadArtifact --> Error
GetState --> Error
Error -->|Yes| SkipOrFail
Error -->|No| Replace
SkipOrFail --> ForEach
Replace --> ForEach
ForEach -->|No More Placeholders| End

Design Patterns and Approaches

Template-Based Injection: Uses simple placeholder syntax with curly braces, enabling clear and flexible instruction authoring.
Lazy Evaluation: Session state and artifact data are fetched only when processing each placeholder, avoiding unnecessary data loading.
Error Tolerance with Optional Placeholders: Supports graceful degradation when optional data is missing, enhancing robustness.
Contextual Abstraction: Relies on the InvocationContext abstraction to access session and artifact services transparently, decoupling implementation details.
Extensibility: The use of instruction providers (InstructionProvider and GlobalInstructionProvider) allows dynamic generation of instructions beyond static templates.

Relationship to Other Topics

The template processing is a critical component of the LLM Integration and Agents topic, as it prepares prompts sent to the LLM.
It depends on session state management provided by the Session Management topic to retrieve current user and app state.
The module also interacts with the Artifact Management topic to load artifact contents dynamically.
The injection mechanism is used in conjunction with agents’ instruction handling and is integrated into the invocation lifecycle managed by the Agent Execution Runner.
The injection logic is encapsulated in internal and utility packages, ensuring its use is consistent across various agents and workflows.

For more detailed understanding of session state handling, see Session State Injection.

The Instruction Template Processing module thus enables powerful and dynamic prompt construction by integrating live session and artifact data, facilitating context-aware AI agent interactions.