In-Memory Session Service

Purpose

The In-Memory Session Service provides a thread-safe, volatile implementation of the session management backend described in the parent topic Session Management. It addresses the need for a lightweight, fast, and concurrency-safe session storage mechanism without relying on external databases or disk persistence.

This subtopic specifically caters to scenarios such as:

• Development and testing environments where persistence isn't required.

• High-performance use cases where the overhead of database interaction is undesirable.

• Situations requiring quick prototyping with session state and event history.

Unlike the database-backed session service, this in-memory service maintains session state and event history entirely within process memory, ensuring rapid access and modification while supporting complex features like event-driven state updates and multi-level state merging.

Functionality

Core Responsibilities

• Session Lifecycle Management: Create, retrieve, list, and delete sessions identified by appName, userID, and sessionID.

• State Persistence: Maintain session-level state with concurrent-safe read/write access.

• Event Handling: Append session events that can include state deltas, with automatic state mutation based on events.

• Multi-level State Merging: Merge session-specific state with user-level and application-level states, providing a layered state model.

• Thread Safety: Use read-write mutexes (sync.RWMutex) to ensure concurrent access to sessions and internal state maps is safe.

Key Workflows and Methods

Session Creation (Create)

• Validates essential identifiers (appName, userID).

• Generates a UUID if no sessionID is provided.

• Initializes session state and merges in application and user state layers.

• Stores the new session in an ordered map keyed by a composite encoded string.

• Returns a copy of the created session with cloned state and event slices.

Session Retrieval (Get)

• Validates input keys.

• Applies filters on events, such as returning only recent events or events after a timestamp.

• Merges session state with user and application states at read time.

• Returns an immutable copy of the session with filtered events.

Session Listing (List)

• Lists all sessions for a given application, optionally filtered by user.

• Uses lexicographical range scans on the ordered map keys.

• Merges state layers for each session before returning.

Event Appending (AppendEvent)

• Validates session and event presence.

• Ignores partial events (streaming partial LLM responses).

• Updates session state by applying deltas from event actions, excluding temporary keys (prefixed with "temp:").

• Updates application and user-level states if deltas are present.

• Appends the event to the session's event list.

• Updates the session's last updated timestamp.

State Management

The service maintains three state layers:

1. Application State: Shared across all sessions and users for an application.

2. User State: Specific to a user within an application.

3. Session State: Specific to an individual session.

State deltas from events are extracted and merged appropriately into these layers, enabling flexible and reusable state management.

Internal Data Structures

• sessions: An ordered map storing session pointers indexed by a composite key (appName, userID, sessionID).

• appState: Map of application-wide aggregated state.

• userState: Nested map keyed by appName and userID for user-specific state.

• session: Internal struct holding event slices, state map, and mutex for concurrent access.

Example Code Snippet

Appending an event updates session and global states:

func (s *inMemoryService) AppendEvent(ctx context.Context, curSession Session, event *Event) error {
    if event.Partial {
        return nil // partial events not persisted
    }
    s.mu.Lock()
    defer s.mu.Unlock()

    storedSession, ok := s.sessions.Get(curSession.(*session).id.Encode())
    if !ok {
        return fmt.Errorf("session not found")
    }

    if err := curSession.(*session).appendEvent(event); err != nil {
        return err
    }

    storedSession.events = append(storedSession.events, event)
    storedSession.updatedAt = event.Timestamp

    if len(event.Actions.StateDelta) > 0 {
        appDelta, userDelta, sessionDelta := sessionutils.ExtractStateDeltas(event.Actions.StateDelta)
        s.updateAppState(appDelta, curSession.AppName())
        s.updateUserState(userDelta, curSession.AppName(), curSession.UserID())
        // merge session delta into session state
        for k, v := range sessionDelta {
            storedSession.state[k] = v
        }
    }
    return nil
}

This method ensures that state changes propagate correctly across layers and that the session event history remains consistent.

Integration with Session Management

The In-Memory Session Service is one of the pluggable storage backends under the parent topic Session Management. It implements the same core interface as the database-backed session service, allowing seamless substitution depending on deployment needs.

• Complementary to Database Session Service: While the database implementation offers durability and transactional guarantees, the in-memory service prioritizes speed and simplicity.

• Event and State Model: Both implementations support event-driven state updates and layered state merging, preserving consistent session semantics.

• Supports Agent Execution and Runner: Session instances retrieved or updated by the in-memory service integrate with the Agent Execution Runner to manage session state during agent invocations.

• Enables State Injection: The session state maintained here can be injected into LLM instructions as described in Instruction Template Processing.

Diagram: Class Structure of In-Memory Session Service

classDiagram
class inMemoryService {
- mu: sync.RWMutex
- sessions: omap.Map[string,*session]
- userState: map[string]map[string]stateMap
- appState: map[string]stateMap
+ Create()
+ Get()
+ List()
+ Delete()
+ AppendEvent()
+ updateAppState()
+ updateUserState()
+ mergeStates()
}
class session {
- id: id
- mu: sync.RWMutex
- events: []*Event
- state: map[string]any
- updatedAt: time.Time
+ ID()
+ AppName()
+ UserID()
+ State()
+ Events()
+ LastUpdateTime()
+ appendEvent()
}
class id {
- appName: string
- userID: string
- sessionID: string
+ Encode()
+ Decode()
}
inMemoryService "1" o-- "*" session : manages
session "1" *-- "1" id : identifies

This diagram illustrates the core structural relationships of the in-memory service, highlighting how sessions are identified, stored, and protected for concurrent access.