llm-hooks.tsx

Overview

The llm-hooks.tsx file provides a collection of React custom hooks and utility functions designed to interact with Large Language Model (LLM) related data in the application. It primarily manages fetching, manipulating, and presenting LLM model lists, factories, and user-specific LLM configurations through asynchronous queries and mutations using React Query (@tanstack/react-query). The hooks facilitate seamless integration with the backend services (via userService) to enable CRUD operations, selection options preparation for UI components, and state synchronization.

This file acts as a bridge between the UI components and the backend API endpoints related to LLMs, enabling components to easily consume and manipulate LLM data through simple hooks encapsulating complex logic and side effects. It also includes utility functions to format LLM options with icons for UI select components.

Detailed Explanation of Exports

1. `useFetchLlmList(modelType?: LlmModelType): IThirdAiModelCollection`

Purpose: Fetches a list of third-party LLM models filtered optionally by model type.
Parameters:
- modelType (optional): A value from the enum LlmModelType to filter the models.
Returns: An object mapping factory names to arrays of third-party LLM models (IThirdAiModelCollection).

Usage Example:

const llmList = useFetchLlmList(LlmModelType.Chat);

Implementation Details: Uses React Query's useQuery to fetch data from userService.llm_list with the optional model_type parameter.

2. `useSelectLlmOptions()`

Purpose: Creates LLM selection options for UI components without icons.
Returns: An array of grouped options formatted as { label: string, options: Array<{ label:string, value:string, disabled: boolean }> }.
Usage Example:
```
const options = useSelectLlmOptions();
```
Implementation Details: Maps the fetched LLM info into grouped options using getRealModelName for labels and disables unavailable models.

3. `useSelectLlmOptionsByModelType()`

Purpose: Provides grouped LLM selection options by different LlmModelTypes, including icons.
Returns: An object mapping each LlmModelType to grouped select options that include icons and disabled state.

Usage Example:

const optionsByType = useSelectLlmOptionsByModelType();
const chatOptions = optionsByType[LlmModelType.Chat];

Implementation Details: Uses filtering based on model_type and tags, then maps models to options with icons using buildLlmOptionsWithIcon.
Key Helper Function: buildLlmOptionsWithIcon(x: IThirdOAIModel) constructs option objects with an icon and label.

4. `useComposeLlmOptionsByModelTypes(modelTypes: LlmModelType[])`

Purpose: Combines multiple model-type-specific LLM options into a single consolidated list, merging options with the same labels.
Parameters:
- modelTypes: Array of LlmModelType enums to compose.
Returns: Array of grouped options compatible with Ant Design Select components.

Usage Example:

const options = useComposeLlmOptionsByModelTypes([LlmModelType.Chat, LlmModelType.Embedding]);

Implementation Details: Uses array reduce to merge grouped options by label.

5. `useFetchLlmFactoryList(): ResponseGetType<IFactory[]>`

Purpose: Fetches the list of LLM factories (providers).
Returns: Object containing factory data array and loading state.

Usage Example:

const { data: factories, loading } = useFetchLlmFactoryList();

Implementation Details: Uses React Query with userService.factories_list.

6. `useFetchMyLlmList(): ResponseGetType<Record<string, IMyLlmValue>>`

Purpose: Fetches the user's configured LLM list.
Returns: Object mapping factory names to user LLM values and loading state.

Usage Example:

const { data: myLlmList, loading } = useFetchMyLlmList();

Implementation Details: Uses React Query with userService.my_llm().

7. `useFetchMyLlmListDetailed(): ResponseGetType<Record<string, any>>`

Purpose: Fetches detailed user LLM information including extra details.
Returns: Object with detailed user LLM data and loading state.

Usage Example:

const { data, loading } = useFetchMyLlmListDetailed();

Implementation Details: Calls userService.my_llm({ include_details: true }).

8. `useSelectLlmList()`

Purpose: Combines user LLM list and factory list into structured lists suitable for UI, including sorting and logo assignment.
Returns: Object with:
- myLlmList: Array of user LLMs enriched with factory logos.
- factoryList: Array of factories not owned by the user, sorted.
- loading: Combined loading state.

Usage Example:

const { myLlmList, factoryList, loading } = useSelectLlmList();

Implementation Details: Uses useMemo for efficient recomputation; uses sortLLmFactoryListBySpecifiedOrder.

9. Mutation Hooks for CRUD Operations on LLM and Factory

All mutation hooks use React Query's useMutation and userService API calls. They handle success messaging and query invalidations to refresh cached data.

Interfaces:
- IApiKeySavingParams: Parameters for saving an API key.
- ISystemModelSettingSavingParams: Parameters for saving tenant system model settings.

a) `useSaveApiKey()`

Purpose: Saves or updates an API key for an LLM factory.
Returns: { data, loading, saveApiKey }

Usage:

const { saveApiKey } = useSaveApiKey();
saveApiKey({ llm_factory: 'openai', api_key: 'xyz' });

Details: Invalidates LLM lists and factory list queries on success.

b) `useSaveTenantInfo()`

Purpose: Saves tenant-specific system model settings.
Returns: { data, loading, saveTenantInfo }

Usage:

const { saveTenantInfo } = useSaveTenantInfo();
saveTenantInfo({ tenant_id: 'id', asr_id: 'asr1', embd_id: 'embd1', img2txt_id: 'img1', llm_id: 'llm1' });

c) `useAddLlm()`

Purpose: Adds a new LLM configuration.
Returns: { data, loading, addLlm }

Usage:

const { addLlm } = useAddLlm();
addLlm({ factory: 'openai', llm_name: 'gpt-4', api_key: 'key' });

d) `useDeleteLlm()`

Purpose: Deletes an existing LLM configuration.
Returns: { data, loading, deleteLlm }

Usage:

const { deleteLlm } = useDeleteLlm();
deleteLlm({ factory: 'openai' });

e) `useDeleteFactory()`

Purpose: Deletes an entire LLM factory.
Returns: { data, loading, deleteFactory }

Usage:

const { deleteFactory } = useDeleteFactory();
deleteFactory({ factory: 'openai' });

Important Implementation Details

React Query Integration: The file extensively uses React Query's useQuery for data fetching and useMutation for modifying data. Cache invalidation strategies are applied to keep UI state consistent.
Internationalization: Success and deletion messages use react-i18next for translations.
UI Integration: Options returned by hooks are designed for use with Ant Design Select components, including disabled states and custom labels with icons.
Data Structure Mapping: The file normalizes and combines data from different API endpoints (factories, user LLMs, third-party LLMs) for convenient UI consumption.
Performance: Uses useMemo and useCallback hooks to optimize rendering and avoid unnecessary recomputations.
Utility Functions: buildLlmOptionsWithIcon creates JSX elements with SVG icons for better UI representation.

Interaction with Other System Components

Backend API (userService): This file depends on userService methods such as llm_list, factories_list, my_llm, add_llm, delete_llm, set_api_key, set_tenant_info, and deleteFactory to perform server-side operations.
UI Components: Provides selection options and mutation hooks for UI components managing LLM models, factories, and user configurations.
Constants and Interfaces: Utilizes enums (LlmModelType), interfaces (IThirdOAIModel, IFactory, etc.), and utility functions (getRealModelName, getLLMIconName) from other parts of the application.
Localization: Uses react-i18next to provide localized messages on mutation success/failure.
Ant Design: Uses Ant Design components (Flex, message, Select option types) for UI elements.

Visual Diagram: Component Diagram of Hook Functions and Data Flow

flowchart TD
    subgraph Fetchers
        direction TB
        useFetchLlmList --> userService.llm_list
        useFetchLlmFactoryList --> userService.factories_list
        useFetchMyLlmList --> userService.my_llm
        useFetchMyLlmListDetailed --> userService.my_llm
    end

    subgraph Selectors
        useSelectLlmOptions --> useFetchLlmList
        useSelectLlmOptionsByModelType --> useFetchLlmList
        useComposeLlmOptionsByModelTypes --> useSelectLlmOptionsByModelType
        useSelectLlmList --> useFetchMyLlmList
        useSelectLlmList --> useFetchLlmFactoryList
    end

    subgraph Mutations
        useSaveApiKey --> userService.set_api_key
        useSaveTenantInfo --> userService.set_tenant_info
        useAddLlm --> userService.add_llm
        useDeleteLlm --> userService.delete_llm
        useDeleteFactory --> userService.deleteFactory
    end

    userService.llm_list -->|returns| useFetchLlmList
    userService.factories_list -->|returns| useFetchLlmFactoryList
    userService.my_llm -->|returns| useFetchMyLlmList
    userService.my_llm -->|returns| useFetchMyLlmListDetailed

    useFetchLlmList --> useSelectLlmOptions
    useFetchLlmList --> useSelectLlmOptionsByModelType
    useSelectLlmOptionsByModelType --> useComposeLlmOptionsByModelTypes
    useFetchMyLlmList --> useSelectLlmList
    useFetchLlmFactoryList --> useSelectLlmList

    Mutations -.->|invalidate queries| useFetchMyLlmList
    Mutations -.->|invalidate queries| useFetchMyLlmListDetailed
    Mutations -.->|invalidate queries| useFetchLlmFactoryList

Summary

This file encapsulates all LLM-related data fetching, caching, and mutation logic for the application.
It provides convenient hooks that abstract API calls and data transformations, enabling UI components to focus on rendering and interaction.
The hooks support filtering, grouping, and formatting LLM options with icons for better UX.
Mutation hooks handle adding, updating, and deleting LLMs and factories, with cache invalidation to keep data fresh.
The file integrates with various utilities and constants to maintain consistency and localization.

This modular design promotes maintainability, testability, and scalability of the LLM management features within the system.

llm-hooks.tsx

Overview

Detailed Explanation of Exports

1. useFetchLlmList(modelType?: LlmModelType): IThirdAiModelCollection

2. useSelectLlmOptions()

3. useSelectLlmOptionsByModelType()

4. useComposeLlmOptionsByModelTypes(modelTypes: LlmModelType[])

5. useFetchLlmFactoryList(): ResponseGetType<IFactory[]>

6. useFetchMyLlmList(): ResponseGetType<Record<string, IMyLlmValue>>

7. useFetchMyLlmListDetailed(): ResponseGetType<Record<string, any>>

8. useSelectLlmList()

9. Mutation Hooks for CRUD Operations on LLM and Factory

a) useSaveApiKey()

b) useSaveTenantInfo()

c) useAddLlm()

d) useDeleteLlm()

e) useDeleteFactory()

Important Implementation Details

Interaction with Other System Components

Visual Diagram: Component Diagram of Hook Functions and Data Flow

Summary

1. `useFetchLlmList(modelType?: LlmModelType): IThirdAiModelCollection`

2. `useSelectLlmOptions()`

3. `useSelectLlmOptionsByModelType()`

4. `useComposeLlmOptionsByModelTypes(modelTypes: LlmModelType[])`

5. `useFetchLlmFactoryList(): ResponseGetType<IFactory[]>`

6. `useFetchMyLlmList(): ResponseGetType<Record<string, IMyLlmValue>>`

7. `useFetchMyLlmListDetailed(): ResponseGetType<Record<string, any>>`

8. `useSelectLlmList()`

a) `useSaveApiKey()`

b) `useSaveTenantInfo()`

c) `useAddLlm()`

d) `useDeleteLlm()`

e) `useDeleteFactory()`