service.go

Overview

The service.go file implements a Google Cloud Storage (GCS) backed artifact storage service conforming to the [artifact.Service](80557) interface. It provides persistent, versioned storage and retrieval of artifacts (binary or textual data) organized by application name, user ID, session ID, filename, and version. This implementation supports both session-scoped and user-scoped artifacts, with user-scoped files identified by a "user:" prefix and stored under a dedicated user namespace.

The service facilitates saving new artifact versions, loading specific or the latest versions, listing available artifact filenames, deleting individual or all versions, and enumerating artifact versions. It uses the official GCS client library, abstracted via wrapper interfaces for testability.

Types and Core Structs

gcsService

The primary struct implementing [artifact.Service](80557) backed by GCS.

type gcsService struct {
	bucketName    string
	storageClient gcsClient
	bucket        gcsBucket
}

• bucketName: The GCS bucket name used for artifact storage.

• storageClient: Wrapped GCS client interface allowing mock implementations.

• bucket: Represents the GCS bucket scoped to the service.

Construction

NewService

Creates a new GCS artifact service instance.

func NewService(ctx context.Context, bucketName string, opts ...option.ClientOption) (artifact.Service, error)

• Parameters:

  • ctx: Context for controlling request lifecycle.

  • bucketName: GCS bucket name string.

  • opts: Optional GCS client options (e.g., credentials).

• Returns:

  • An initialized artifact.Service implementation.

  • An error if client creation fails.

Usage Example:

svc, err := NewService(ctx, "my-gcs-bucket", option.WithCredentialsFile("path/to/creds.json"))
if err != nil {
    // handle error
}
// svc can now be used to Save, Load, Delete, List, Versions

Naming and Scoping Helpers

Several helper functions build GCS blob/object names for versioned artifact storage:

• fileHasUserNamespace(filename string) bool
  Returns true if the filename indicates a user-scoped namespace (prefix "user:").

• buildBlobName(appName, userID, sessionID, fileName string, version int64) string
  Constructs the full blob path including version number.

• buildBlobNamePrefix(appName, userID, sessionID, fileName string) string
  Constructs the blob path prefix (without version).

• buildSessionPrefix(appName, userID, sessionID string) string
  Constructs the prefix for all blobs in a session.

• buildUserPrefix(appName, userID string) string
  Constructs the prefix for user-scoped blobs.

Blob Naming Logic:

• For session-scoped artifacts:
  {appName}/{userID}/{sessionID}/{fileName}/{version}

• For user-scoped artifacts (filename prefixed with "user:"):
  {appName}/{userID}/user/{fileName}/{version}

Methods Implementing artifact.Service

Save

func (s *gcsService) Save(ctx context.Context, req *artifact.SaveRequest) (*artifact.SaveResponse, error)

• Purpose: Saves a new version of an artifact to GCS.

• Parameters: SaveRequest containing AppName, UserID, SessionID, FileName, and Part (artifact content).

• Returns: SaveResponse with the assigned version number or an error.

• Details:

  • Validates the request.

  • Lists existing versions to determine the next version number (nextVersion).

  • Constructs the blob name with buildBlobName.

  • Opens a GCS writer for the blob.

  • Writes inline binary data or textual content based on Part.

  • Sets appropriate MIME type.

  • Closes the writer and returns the new version.

Important Notes:

• There is a known race condition when multiple clients save concurrently, due to lack of transactions in GCS.

• Version numbers always increment starting at 1.

Load

func (s *gcsService) Load(ctx context.Context, req *artifact.LoadRequest) (*artifact.LoadResponse, error)

• Purpose: Loads an artifact by version or latest version if none specified.

• Parameters: LoadRequest with identifying fields and optional version.

• Returns: LoadResponse containing a genai.Part with the artifact data.

• Details:

  • Validates the request.

  • If version is 0, fetches all versions and picks the highest.

  • Constructs blob name for the requested version.

  • Checks if the blob exists, returns fs.ErrNotExist if missing.

  • Opens a GCS reader and reads all data.

  • Creates a genai.Part from bytes and MIME type.

  • Returns the loaded artifact part.

Delete

func (s *gcsService) Delete(ctx context.Context, req *artifact.DeleteRequest) error

• Purpose: Deletes a specific version or all versions of an artifact.

• Parameters: DeleteRequest including artifact identity and optional version.

• Behavior:

  • Validates request.

  • If a specific version is requested, deletes that single blob.

  • If version is 0, fetches all versions and deletes them in parallel using errgroup.

• Concurrency: Parallel deletion uses goroutines with context cancellation propagation.

• Error Handling: Returns errors if deletion fails; no error if artifact does not exist.

List

func (s *gcsService) List(ctx context.Context, req *artifact.ListRequest) (*artifact.ListResponse, error)

• Purpose: Lists all artifact filenames available in a session and user namespace.

• Details:

  • Validates request.

  • Fetches filenames under the session prefix.

  • Fetches filenames under the user prefix.

  • Merges filenames into a set to avoid duplicates.

  • Returns sorted list of filenames.

Versions

func (s *gcsService) Versions(ctx context.Context, req *artifact.VersionsRequest) (*artifact.VersionsResponse, error)

• Purpose: Returns all versions of a given artifact.

• Details:

  • Calls internal versions helper to list versions based on blob listings.

  • Returns error if no versions found (fs.ErrNotExist).

Internal and Helper Functions

versions

Internal method listing versions for an artifact without error if none found:

func (s *gcsService) versions(ctx context.Context, req *artifact.VersionsRequest) (*artifact.VersionsResponse, error)

• Queries GCS blobs with prefix.

• Parses version numbers from blob names.

• Ignores non-numeric versions.

• Returns slice of versions.

fetchFilenamesFromPrefix

Helper to fetch artifact filenames under a given GCS prefix:

func (s *gcsService) fetchFilenamesFromPrefix(ctx context.Context, prefix string, filenamesSet map[string]bool) error

• Iterates blobs under prefix.

• Extracts the filename segment before version.

• Adds to filenamesSet.

• Handles user and session namespaces.

Important Implementation Details

• Versioning:
  Versions are managed by enumerating existing blobs and picking the next highest version number, since GCS does not provide transaction support or atomic increments.

• Blob Naming:
  The hierarchical blob naming convention encodes artifact identity and version, enabling efficient prefix queries for listing and version enumeration.

• User Namespace Support:
  Files prefixed with "user:" are stored under the {app}/{user}/user/ prefix, making artifacts accessible across sessions for that user.

• Parallel Deletion:
  Uses errgroup to delete multiple versions concurrently, improving performance for bulk deletions.

• Content Handling:
  Supports inline binary data with MIME type or plain text, setting the GCS blob content type accordingly.

• Error Handling:
  Attempts to return precise errors, e.g., fs.ErrNotExist for missing artifacts, and wraps errors with context messages.

• Client Wrapping:
  The GCS client is wrapped by interfaces (gcsClient, gcsBucket, gcsObject) allowing mocking in tests (gcs_test.go).

Interaction with Other Parts of the System

• Implements the artifact.Service interface, enabling usage by agents, tools, and session management components to persist artifacts.

• Artifacts saved via this service can be loaded or listed by agents during workflows, including via tools such as the Artifact Loading Tool.

• The file naming and versioning scheme aligns with the broader artifact management conventions to maintain consistency across different backends.

• The service is a critical persistent backend alongside the in-memory implementation, supporting durable storage in production environments.

• Integrates with Google Cloud authentication and clients, relying on cloud.google.com/go/storage and Google API options.

Visual Diagram

classDiagram
class gcsService {
-bucketName: string
-storageClient: gcsClient
-bucket: gcsBucket
+Save()
+Load()
+Delete()
+List()
+Versions()
}
class blobNameHelpers {
+fileHasUserNamespace()
+buildBlobName()
+buildBlobNamePrefix()
+buildSessionPrefix()
+buildUserPrefix()
}
gcsService ..> blobNameHelpers : uses

Detailed Function and Method Descriptions

NewService

• Creates a new GCS artifact service instance.

• Calls storage.NewClient with provided options.

• Wraps the client in gcsClientWrapper to abstract the GCS client interface.

• Prepares the bucket interface.

• Returns *gcsService.

fileHasUserNamespace(filename string) bool

• Checks if the filename starts with "user:".

• Used to distinguish user-scoped blobs.

buildBlobName(appName, userID, sessionID, fileName string, version int64) string

• Constructs the full blob path with version.

• For user-scoped files, uses /user/ namespace without session ID.

• For session-scoped files, uses the full path including session ID.

(s *gcsService) Save(ctx context.Context, req *artifact.SaveRequest) (*artifact.SaveResponse, error)

• Validates the save request.

• Calls internal versions to get existing versions.

• Determines nextVersion by incrementing max existing version or 1 if none.

• Builds blob name for new version.

• Opens GCS writer.

• Writes inline binary or text data.

• Returns assigned version.

(s *gcsService) Delete(ctx context.Context, req *artifact.DeleteRequest) error

• Validates the delete request.

• Deletes specific version if provided.

• Otherwise deletes all versions concurrently.

• Uses errgroup for concurrency.

• Returns error on failure.

(s *gcsService) Load(ctx context.Context, req *artifact.LoadRequest) (*artifact.LoadResponse, error)

• Validates the load request.

• If version 0, fetches latest version.

• Builds blob name.

• Gets blob attributes (to check existence and content-type).

• Creates a reader.

• Reads all content.

• Constructs genai.Part from bytes and content type.

• Returns artifact part.

(s *gcsService) List(ctx context.Context, req *artifact.ListRequest) (*artifact.ListResponse, error)

• Validates request.

• Collects filenames from session prefix.

• Collects filenames from user prefix.

• Deduplicates and sorts filenames.

• Returns list of filenames.

(s *gcsService) Versions(ctx context.Context, req *artifact.VersionsRequest) (*artifact.VersionsResponse, error)

• Calls internal versions.

• Returns error if no versions found.

• Otherwise returns list of versions.

(s *gcsService) versions(ctx context.Context, req *artifact.VersionsRequest) (*artifact.VersionsResponse, error)

• Validates request.

• Queries blobs with prefix.

• Parses version numbers from blob names.

• Returns versions slice.

(s *gcsService) fetchFilenamesFromPrefix(ctx context.Context, prefix string, filenamesSet map[string]bool) error

• Queries blobs under prefix.

• Extracts filenames from blob paths.

• Adds filenames to set.

Usage Example

ctx := context.Background()
service, err := gcs.NewService(ctx, "my-bucket")
if err != nil {
    log.Fatal(err)
}

// Save new artifact
saveResp, err := service.Save(ctx, &artifact.SaveRequest{
    AppName: "myapp",
    UserID: "user123",
    SessionID: "sess456",
    FileName: "file.txt",
    Part: genai.NewPartFromText("Hello World"),
})
if err != nil {
    log.Fatal(err)
}
fmt.Println("Saved version:", saveResp.Version)

// Load latest artifact version
loadResp, err := service.Load(ctx, &artifact.LoadRequest{
    AppName: "myapp",
    UserID: "user123",
    SessionID: "sess456",
    FileName: "file.txt",
})
if err != nil {
    log.Fatal(err)
}
fmt.Println("Loaded content:", loadResp.Part.Text)

This file is a foundational implementation for durable, versioned artifact storage in GCS, enabling agents and workflows in the system to reliably persist and retrieve artifacts with scoped naming and concurrency-aware deletion. It complements other backend implementations and integrates with the overall Artifact Management infrastructure.