load_artifacts_tool_test.go

Overview

This file contains unit tests for the Artifact Loading Tool implemented in the loadartifactstool package. The tests validate the tool's core functionalities, including argument handling, artifact loading, and request processing within an agent invocation context. The tool enables AI agents or LLMs to load and access artifacts dynamically during execution by exposing a function call interface and manipulating LLM requests.

The tests ensure that the tool correctly:

• Parses and validates input arguments.

• Returns expected outputs or errors for various input scenarios.

• Injects system instructions about available artifacts into LLM requests.

• Processes function calls to load artifact contents and appends them appropriately.

• Handles requests involving other function calls gracefully without altering content.

This test file exercises the tool's integration with artifact storage and the agent invocation context to simulate realistic usage scenarios.

Detailed Explanations

Test Functions

1. TestLoadArtifactsTool_Run

• Purpose:
  Tests the Run method of the loadartifactstool's internal function tool interface. It verifies argument parsing, error handling, and output correctness.

• Setup:

  • Creates a new instance of the load artifacts tool.

  • Converts it to the toolinternal.FunctionTool interface to invoke Run.

  • Establishes a tool context with an in-memory artifact service.

• Test Cases:
  Covers scenarios including:

  • Basic string slice input.

  • Empty arguments or slices.

  • Inputs as slices of any with string elements.

  • Nil input for artifact names.

  • Incorrect type inputs (non-slice or containing non-string elements).

• Parameters:

  • args (map[string]any): Input arguments, expected to contain "artifact_names" as a slice of strings or any.

• Return Values:

  • map[string]any: Returns a normalized map with "artifact_names" as a []string.

  • error: Returns error if the input type is invalid or contains non-string elements.

• Usage Example:

  result, err := toolImpl.Run(tc, map[string]any{
      "artifact_names": []string{"file1", "file2"},
  })
  

• Validation:
  Uses cmp.Diff to compare expected and actual output maps.

2. TestLoadArtifactsTool_ProcessRequest

• Purpose:
  Tests the ProcessRequest method of the load artifacts tool, verifying that it correctly injects system instructions listing available artifacts into an LLM request.

• Setup:

  • Creates an in-memory artifact context.

  • Saves two artifacts named "file1.txt" and "file2.pdf".

  • Creates an empty model.LLMRequest.

  • Confirms that the tool implements toolinternal.RequestProcessor.

• Functionality Tested:

  • After ProcessRequest, the system instruction of the LLMRequest contains a message informing about available artifacts.

  • The instruction text should mention "You have a list of artifacts" and include the artifact names.

  • Ensures no contents are appended at this stage.

• Parameters:

  • tc (tool.Context): The invocation context with artifact access.

  • llmRequest (*model.LLMRequest): The LLM request to be processed.

• Return Value:

  • error: Returns error if the processing fails.

3. TestLoadArtifactsTool_ProcessRequest_Artifacts_LoadArtifactsFunctionCall

• Purpose:
  Validates that when the LLM request contains a function call to "load_artifacts" with artifact names, the tool loads the artifact contents and appends them as user content.

• Setup:

  • Saves a single artifact "doc1.txt" with content.

  • Constructs an LLMRequest with a content part representing a function response calling "load_artifacts" for "doc1.txt".

• Test Logic:

  • Calls ProcessRequest to process the function call.

  • Verifies that the LLMRequest.Contents is appended with a new user role content that includes:

    • A text part indicating "Artifact doc1.txt is:".

    • The actual content of the artifact.

• Assertions:

  • The number of contents after processing is 2 (model + user).

  • The appended content role is "user".

  • The appended content parts match the expected texts.

4. TestLoadArtifactsTool_ProcessRequest_Artifacts_OtherFunctionCall

• Purpose:
  Ensures that function calls other than "load_artifacts" do not cause the tool to load artifacts or modify the LLMRequest contents.

• Setup:

  • Saves an artifact "doc1.txt".

  • Builds an LLMRequest containing a function call with a different name, e.g., "other_function".

• Validation:

  • After processing, the contents remain unchanged with only the original model content.

  • The content role remains "model".

Helper Function

createToolContext(t *testing.T) tool.Context

• Purpose:
  Creates a tool.Context configured with an in-memory artifact service and initialized session/app parameters for testing.

• Usage:
  Used by tests to provide a consistent invocation context with artifacts and context for tool execution.

• Implementation Details:

  • Creates an artifactinternal.Artifacts instance backed by artifact.InMemoryService().

  • Builds an invocation context (icontext.InvocationContext) that embeds the artifacts.

  • Returns a toolinternal.ToolContext wrapping the invocation context.

Important Implementation Details and Algorithms

• Type Safety in Argument Parsing:
  The Run method rigorously checks the type of the "artifact_names" argument, allowing both []string and []any with string elements. It returns an error for mismatches or invalid types.

• Concurrent Artifact Loading:
  Although not explicitly shown in this test file, the ProcessRequest method in the actual tool implementation uses concurrency to load multiple artifacts efficiently, as can be inferred from the testing of multiple artifact names.

• Function Call Detection:
  The tool differentiates function calls by name ("load_artifacts" vs others) and selectively processes only relevant function calls to load artifacts.

• LLMRequest Mutation:
  The tool modifies the LLMRequest by appending system instructions or content parts, influencing the prompt sent to the LLM or subsequent generation steps.

• Integration with Internal APIs:
  The tests leverage internal packages such as toolinternal, artifactinternal, and icontext to create rich contexts that mimic real agent execution environments.

Interactions with Other System Components

• Artifact Service (artifact and artifactinternal):
  Provides storage and retrieval capabilities for named artifacts. The tool accesses this service through the invocation context to read artifact contents.

• Tooling System (tool and toolinternal):
  The load artifacts tool implements interfaces like toolinternal.FunctionTool and toolinternal.RequestProcessor, integrating into the tooling framework that agents use to invoke tools and process LLM requests.

• Agent Invocation Context (icontext):
  Supplies session-scoped contextual data including artifacts. The tool operates within this context to access user/session-specific artifacts.

• LLM Request and Model (model and genai):
  The tool modifies LLM requests by appending system instructions and content parts, affecting how the LLM generates text or function calls.

Mermaid Diagram: Flow of LoadArtifactsTool Test Functions and Interactions

flowchart TD
A[Start TestLoadArtifactsTool_Run] --> B[Create LoadArtifactsTool Instance]
B --> C[Create ToolContext with Artifacts]
C --> D["Invoke Run() with Args"]
D --> E{Args Valid?}
E -- Yes --> F[Return Normalized Artifact Names]
E -- No --> G[Return Error]
F --> H[Compare Result with Expected]
H --> I[End TestLoadArtifactsTool_Run]
J[Start TestLoadArtifactsTool_ProcessRequest] --> K[Create LoadArtifactsTool Instance]
K --> L[Save Artifacts to Context]
L --> M[Create Empty LLMRequest]
M --> N["Invoke ProcessRequest()"]
N --> O[Inject System Instruction with Artifact List]
O --> P[Verify Instruction Content]
P --> Q[End TestLoadArtifactsTool_ProcessRequest]
R[Start TestLoadArtifactsTool_ProcessRequest_Artifacts_LoadArtifactsFunctionCall] --> S[Save Artifact]
S --> T[Create LLMRequest with "load_artifacts" FunctionCall]
T --> U["ProcessRequest()"]
U --> V[Load Artifact Content and Append User Content]
V --> W[Verify Appended Content Parts]
W --> X[End TestLoadArtifactsTool_ProcessRequest_Artifacts_LoadArtifactsFunctionCall]
Y[Start TestLoadArtifactsTool_ProcessRequest_Artifacts_OtherFunctionCall] --> Z[Save Artifact]
Z --> AA[Create LLMRequest with Other FunctionCall]
AA --> AB["ProcessRequest()"]
AB --> AC[Verify Content Unchanged]
AC --> AD[End TestLoadArtifactsTool_ProcessRequest_Artifacts_OtherFunctionCall]

References to Related Topics

• The tests exercise the tool defined in the Artifact Loading Tool subtopic, which details the tool's purpose, function declaration, and LLM request processing.

• The tool's usage of artifact services ties into Artifact Management, which governs artifact storage and retrieval interfaces.

• The interaction with LLM requests and function calls relates to the broader Tooling System, which defines the modular tool framework and interfaces.

• The creation and usage of invocation contexts are explained in Agent Invocation Context, providing the environment for tool execution.

This file is essential for validating the correctness and robustness of the loadartifactstool package's artifact loading capabilities, ensuring seamless integration with the agent framework and artifact storage system.