inmemory.go

Overview

The inmemory.go file implements an in-memory, thread-safe memory service that stores and retrieves conversational memory entries associated with user sessions. This service maintains session data such as content from language model (LLM) responses, authorship, timestamps, and precomputed word sets for efficient keyword-based search. It acts as a volatile storage backend designed primarily for transient or testing scenarios where persistent storage is not required.

The service provides two primary operations:

• Adding sessions with event data (AddSession).

• Searching stored memories based on keyword queries (Search).

This file is part of the broader memory management system that enables conversational agents to retain and recall past interactions, tying in closely with session management and LLM event processing as described in Session Management and LLM Integration and Agents.

Types and Structures

key (struct)

• Fields:

  • appName string — The application identifier owning the session.

  • userID string — The user identifier associated with the session.

Used as the primary lookup key in the in-memory store to isolate memory entries per user-app combination.

sessionID (type alias)

• Underlying type: string

Represents a unique session identifier.

value (struct)

• Fields:

  • content *genai.Content — Pointer to the LLM-generated content.

  • author string — Author of the content (e.g., system, user).

  • timestamp time.Time — Time the event occurred.

  • words map[string]struct{} — Precomputed set of normalized words extracted from the content, used for fast keyword matching during search.

Encapsulates a single memory entry connected to a session event.

inMemoryService (struct)

• Fields:

  • mu sync.RWMutex — Mutex guarding concurrent access to the store.

  • store map[key]map[sessionID][]value — Nested map storing memory values per app-user key, session ID, and event values list.

Implements the Service interface to manage in-memory sessions and searchable memory entries in a thread-safe manner.

Functions and Methods

InMemoryService() Service

• Description: Factory function returning a new instance of the in-memory memory service.

• Returns: Service — an interface representing the memory service.

• Usage:

  svc := memory.InMemoryService()
  

• Details: Initializes the nested map structure for storing sessions and their associated memory values.

(s *inMemoryService) AddSession(ctx context.Context, curSession session.Session) error

• Description: Adds a session's event data into the in-memory store.

• Parameters:

  • ctx context.Context — Context for cancellation and deadlines.

  • curSession session.Session — The session object containing event history.

• Returns: error — Non-nil if an error occurs.

• Operation:

  • Iterates over all session events.

  • Extracts LLM response content and author metadata.

  • Builds a words set for each content by splitting text into normalized tokens.

  • Aggregates value structs for each event with relevant metadata.

  • Locks the store for write access, then inserts or updates the nested map keyed by appName, userID, and sessionID.

• Usage Example:

  err := svc.AddSession(ctx, session)
  if err != nil {
      log.Fatal(err)
  }
  

• Notes: Skips events with empty content or empty extracted word sets to optimize storage.

(s *inMemoryService) Search(ctx context.Context, req *SearchRequest) (*SearchResponse, error)

• Description: Searches stored memories matching keywords extracted from the query string.

• Parameters:

  • ctx context.Context — Context for cancellation and deadlines.

  • req *SearchRequest — Search request containing AppName, UserID, and Query string.

• Returns:

  • *SearchResponse — Contains matched memory entries.

  • error — Non-nil if an error occurs.

• Operation:

  • Extracts normalized keywords from the query string.

  • Reads the in-memory map for the given appName and userID.

  • Iterates over all stored session events and matches their precomputed word sets against the query words.

  • Uses checkMapsIntersect helper to determine intersection between keyword sets.

  • Appends matched entries to the response.

• Usage Example:

  resp, err := svc.Search(ctx, &memory.SearchRequest{
      AppName: "myapp",
      UserID: "user123",
      Query: "example query",
  })
  if err != nil {
      log.Fatal(err)
  }
  fmt.Println(resp.Memories)
  

• Notes: Returns empty results if no matching user/app data exists.

checkMapsIntersect(m1, m2 map[string]struct{}) bool

• Description: Utility function to check if two string sets intersect.

• Parameters:

  • m1, m2 map[string]struct{} — Two maps representing sets of normalized words.

• Returns: bool — true if there is at least one common key; otherwise false.

• Algorithm:

  • Early return if either map is empty.

  • Iterates over the smaller map for efficiency.

  • Checks presence of each key in the other map.

• Usage: Used internally by the search method to match query keywords against stored content.

extractWords(text string) map[string]struct{}

• Description: Extracts a set of normalized words from input text.

• Parameters:

  • text string — Input string to tokenize.

• Returns: map[string]struct{} — Set of lowercase words.

• Algorithm:

  • Splits the text on spaces using strings.SplitSeq.

  • Filters out empty tokens.

  • Normalizes tokens to lowercase.

• Usage: Used for indexing content and query strings to enable keyword search.

• Example:

  words := extractWords("Hello World!")
  // words contains "hello", "world!"
  

Important Implementation Details

• Thread Safety: The service uses a sync.RWMutex to allow concurrent reads but exclusive writes to the underlying store, ensuring safe access in concurrent scenarios.

• Data Structure: A three-level nested map structure is used:

  • Outer map keyed by key (appName + userID).

  • Middle map keyed by sessionID.

  • Inner slice holds value entries representing individual events.

• Keyword Matching: To efficiently search memory, each stored content's words are precomputed and stored in a map for O(1) membership checks. The search compares the intersection of sets instead of full-text matching.

• Memory Model: This service is volatile and does not persist data beyond process lifetime. It is suited for quick prototyping or ephemeral usage scenarios.

Interaction with Other System Components

• Session Management: Relies on the session.Session interface from Session Management to obtain event histories.

• Content Representation: Uses genai.Content from the LLM integration package (LLM Integration and Agents) to represent LLM response content.

• Service Interface: Implements the Service interface which is a common abstraction for memory services, allowing this in-memory implementation to be swapped with persistent backends.

• Concurrency Context: Supports context propagation for cancellation and deadline management, aligning with Go idioms for long-running or concurrent operations.

Diagram: inMemoryService Structure and Workflow

classDiagram
class inMemoryService {
-mu: RWMutex
-store: map[key]map[sessionID][]value
+AddSession(ctx, session)
+Search(ctx, req)
}
class key {
+appName: string
+userID: string
}
class sessionID {
<<alias>> string
}
class value {
-content: *genai.Content
-author: string
-timestamp: time.Time
-words: map[string]struct{}
}
inMemoryService "1" o-- "*" key : store keys
key "1" o-- "*" sessionID : sessions per user/app
sessionID "1" o-- "*" value : memory entries per session

This class diagram illustrates the nested data structure of the inMemoryService:

• The store maps from a composite key (app + user) to multiple sessionIDs.

• Each sessionID maps to a slice of value entries representing stored memories.

• The inMemoryService exposes the main methods AddSession and Search to modify and query this structure.

References:

• For session and event lifecycle details, see Session Management.

• For LLM content structure and integration, see LLM Integration and Agents.