dialog_service.py

Overview

dialog_service.py is a core service module in the InfiniFlow system that manages dialog-related functionalities, including dialog record management, chat interactions with large language models (LLMs), knowledge base retrieval, SQL generation for question answering, and mind map generation from knowledge bases. This file integrates several subsystems such as database models, LLM bundles, knowledge retrieval, and Langfuse tracing to provide a comprehensive dialog experience for tenants.

Key responsibilities include:

• CRUD operations for dialog records.

• Orchestrating chat interactions with LLMs, including streaming and non-streaming responses.

• Retrieving and filtering knowledge bases relevant to dialogs.

• Generating SQL for database-backed retrieval when applicable.

• Repairing and formatting citations in generated answers.

• Generating mind maps from retrieved knowledge chunks.

• Supporting Text-To-Speech (TTS) functionality if enabled in dialog configurations.

• Integrating Langfuse for traceability and performance monitoring.

Classes and Functions

Class: DialogService

A subclass of CommonService managing database operations on the Dialog model.

Methods

• save(**kwargs)

  • Purpose: Insert a new dialog record into the database forcibly as a new entry.

  • Parameters: Arbitrary keyword arguments representing dialog fields.

  • Returns: The saved model instance.

  • Usage Example:

    new_dialog = DialogService.save(name="Test Dialog", tenant_id=123, status=1)
    

• update_many_by_id(data_list)

  • Purpose: Batch update multiple dialog records by their IDs, automatically updating timestamps.

  • Parameters:

    • data_list (list of dict): Each dict must contain an id key and other fields to update.

  • Returns: None (performs DB updates).

  • Usage Example:

    DialogService.update_many_by_id([
        {"id": 1, "name": "Updated Name"},
        {"id": 2, "status": StatusEnum.INVALID.value}
    ])
    

• get_list(tenant_id, page_number, items_per_page, orderby, desc, id, name)

  • Purpose: Retrieve a paginated list of dialogs filtered by tenant, optional ID, or name.

  • Parameters:

    • tenant_id (int): Tenant to filter dialogs.

    • page_number (int): Page number for pagination.

    • items_per_page (int): Number of items per page.

    • orderby (str): Field to order by.

    • desc (bool): True for descending order.

    • id (int or None): Optional dialog ID filter.

    • name (str or None): Optional dialog name filter.

  • Returns: List of dictionaries representing dialogs.

  • Usage Example:

    dialogs = DialogService.get_list(tenant_id=123, page_number=1, items_per_page=10, orderby="update_time", desc=True, id=None, name=None)
    

• get_by_tenant_ids(joined_tenant_ids, user_id, page_number, items_per_page, orderby, desc, keywords, parser_id=None)

  • Purpose: Retrieve dialogs for multiple tenants and a user, supporting text search and optional parser filtering.

  • Parameters:

    • joined_tenant_ids (list): List of tenant IDs.

    • user_id (int): User ID for dialog isolation.

    • page_number, items_per_page, orderby, desc: Pagination and ordering.

    • keywords (str): Keyword search on dialog names.

    • parser_id (int or None): Optional parser ID filter.

  • Returns: Tuple (list of dialogs as dicts, total count).

  • Usage Example:

    dialogs, total = DialogService.get_by_tenant_ids([1, 2], 3, 1, 10, "update_time", True, "chat", parser_id=5)
    

Function: chat_solo(dialog, messages, stream=True)

Performs a chat interaction with an LLM without knowledge base integration.

• Parameters:

  • dialog (Dialog model instance): Dialog configuration.

  • messages (list): List of message dicts with role and content.

  • stream (bool): If True, yields partial answers as they stream.

• Returns: Generator yielding dicts with keys:

  • "answer": Text answer.

  • "reference": Empty dict (no references).

  • "audio_binary": Hex-encoded audio if TTS enabled.

  • "prompt": Empty string.

  • "created_at": Timestamp.

• Usage Example:

  for response in chat_solo(dialog, messages, stream=True):
      print(response["answer"])
  

Function: get_models(dialog)

Fetches and prepares various LLM bundles and related knowledge bases for a dialog.

• Parameters:

  • dialog (Dialog model instance).

• Returns: Tuple (kbs, embd_mdl, rerank_mdl, chat_mdl, tts_mdl):

  • kbs: List of Knowledgebase objects.

  • embd_mdl: LLMBundle for embedding model or None.

  • rerank_mdl: LLMBundle for reranker or None.

  • chat_mdl: LLMBundle for chat or image2text model.

  • tts_mdl: LLMBundle for TTS model or None.

• Raises: Exception if multiple embedding models used across KBs.

• Usage Example:

  kbs, embd_mdl, rerank_mdl, chat_mdl, tts_mdl = get_models(dialog)
  

Function: repair_bad_citation_formats(answer: str, kbinfos: dict, idx: set)

Normalizes and repairs citation patterns in the generated answers.

• Parameters:

  • answer (str): Generated answer text.

  • kbinfos (dict): Knowledge base info containing chunks.

  • idx (set): Set of citation indices.

• Returns: Tuple (repaired_answer, updated_idx).

• Details: Uses regex patterns to find and replace inconsistent citation formats like (ID: 12) or ref12 with standardized [ID:12].

• Usage Example:

  fixed_answer, indices = repair_bad_citation_formats(answer, kbinfos, set())
  

Function: convert_conditions(metadata_condition)

Converts metadata filter conditions from UI format to internal operator notation.

• Parameters:

  • metadata_condition (dict or None): Filter conditions with name, comparison_operator, and value.

• Returns: List of dicts with keys: op, key, value.

• Usage Example:

  conditions = convert_conditions(metadata_condition)
  

Function: meta_filter(metas: dict, filters: list[dict])

Filters document IDs based on metadata filters.

• Parameters:

  • metas (dict): Metadata mapping field values to document IDs.

  • filters (list): List of filter dicts with keys op, key, value.

• Returns: List of document IDs that satisfy all filters.

• Usage Example:

  filtered_doc_ids = meta_filter(metas, filters)
  

Function: chat(dialog, messages, stream=True, **kwargs)

Main chatbot function integrating dialogs, knowledge bases, and LLMs.

• Parameters:

  • dialog (Dialog instance).

  • messages (list of dict): Conversation messages.

  • stream (bool): Whether to stream responses.

  • kwargs: Optional parameters like toolcall_session, tools, doc_ids.

• Yields: Dicts with keys:

  • "answer" (str): Answer text.

  • "reference" (dict): Knowledge chunks and doc aggregates.

  • "audio_binary" (str): Hex-encoded audio if TTS enabled.

  • "prompt" (str): Prompt and performance info.

  • "created_at" (float): Timestamp.

• Implementation Details:

  • Detects LLM type and prepares models.

  • Supports Langfuse tracing for observability.

  • Retrieves relevant knowledge bases and documents using embedding and rerank models.

  • Supports reasoning through DeepResearcher.

  • Incorporates SQL-based retrieval if configured.

  • Handles cross-language prompts and metadata filtering.

  • Inserts citations into answers and repairs citation formats.

  • Measures and reports detailed timing and token usage statistics.

• Usage Example:

  for response in chat(dialog, messages, stream=True):
      print(response["answer"])
  

Function: use_sql(question, field_map, tenant_id, chat_mdl, quota=True, kb_ids=None)

Generates and executes SQL queries based on user questions and table metadata.

• Parameters:

  • question (str): User's question.

  • field_map (dict): Maps database fields.

  • tenant_id (int).

  • chat_mdl (LLMBundle): Chat model used to generate SQL.

  • quota (bool): Whether to include quotas in output.

  • kb_ids (list or None): Optional knowledge base filters.

• Returns: Dict with keys:

  • "answer": Markdown table of SQL query results.

  • "reference": Chunks and document aggregations.

  • "prompt": The system prompt used.

  • Or None if query fails.

• Implementation Details:

  • Uses LLM to generate SQL.

  • Sanitizes and validates SQL.

  • Retries if SQL errors occur with error context.

  • Formats results as Markdown table with citations.

• Usage Example:

  sql_response = use_sql(question, field_map, tenant_id, chat_mdl)
  

Function: tts(tts_mdl, text)

Generates TTS audio in hex string format.

• Parameters:

  • tts_mdl (LLMBundle or None): TTS model.

  • text (str): Text to be converted.

• Returns: Hex string of audio binary or None.

• Usage Example:

  audio_hex = tts(tts_mdl, "Hello world")
  

Function: ask(question, kb_ids, tenant_id, chat_llm_name=None, search_config={})

Simple Q&A function using knowledge base retrieval and chat model.

• Parameters:

  • question (str): User question.

  • kb_ids (list): Knowledge base IDs.

  • tenant_id (int).

  • chat_llm_name (str or None): Optional chat LLM name.

  • search_config (dict): Additional search parameters (doc_ids, rerank_id, meta_data_filter, etc.).

• Yields: Dicts with "answer" and "reference".

• Usage Example:

  for response in ask("What is AI?", [1, 2], 123):
      print(response["answer"])
  

Function: gen_mindmap(question, kb_ids, tenant_id, search_config={})

Generates a mind map visualization based on knowledge base documents relevant to a question.

• Parameters:

  • question (str): Query question.

  • kb_ids (list): Knowledge base IDs.

  • tenant_id (int).

  • search_config (dict): Optional config like meta_data_filter, doc_ids, rerank_id.

• Returns: Mind map data structure or error dict.

• Implementation Details:

  • Retrieves documents using embedding and rerank models.

  • Uses MindMapExtractor to process content asynchronously with trio.

• Usage Example:

  mind_map = gen_mindmap("Explain blockchain", [1], 123)
  

Important Implementation Details and Algorithms

• Streaming Chat Responses: The chat and chat_solo functions yield partial outputs as tokens are generated, filtering out short token increments for smoother streaming.

• Knowledge Retrieval: Integrates embedding-based similarity search, reranking, and optionally knowledge graphs or Tavily API results.

• Citation Repair: Uses regex to fix inconsistent citation formats in LLM-generated answers, ensuring references are correctly linked.

• SQL Generation and Execution: Uses LLMs to generate SQL queries based on user questions and table schemas, with retry logic for errors.

• Langfuse Tracing: Implements detailed tracing and timing via Langfuse, recording phases like LLM checking, retrieval, and answer generation.

• Metadata Filtering: Applies user-defined or auto-generated metadata filters on knowledge base documents to narrow down retrieval.

• TTS Integration: Converts answer text into audio hex stream if TTS enabled in dialog prompt config.

• Mind Map Extraction: Uses asynchronous retrieval and NLP-based extraction to create visual mind maps from knowledge chunks.

Interaction with Other System Components

• Database Models: Uses Dialog and related models via Peewee ORM for persistence.

• LLMBundle: Wrapper around various LLM types (chat, embedding, rerank, TTS, image2text).

• KnowledgebaseService & DocumentService: To fetch and filter knowledge base documents.

• TenantLLMService: Provides LLM configuration based on tenant and model IDs.

• Retrievaler: Embedding and rerank retrieval backend for document search.

• Langfuse: External observability and tracing service integration.

• Tavily: Optional external API for document chunk retrieval.

• MindMapExtractor: NLP component for mind map generation.

• Prompts: Uses predefined Jinja-based prompt templates for SQL generation and citations.

• Settings Module: Holds global settings and services like retrievaler.

Visual Diagram

classDiagram
    class DialogService {
        +save(**kwargs)
        +update_many_by_id(data_list)
        +get_list(tenant_id, page, per_page, orderby, desc, id, name)
        +get_by_tenant_ids(joined_tenant_ids, user_id, page, per_page, orderby, desc, keywords, parser_id)
    }
    class LLMBundle {
        +chat(system_prompt, messages, settings)
        +chat_streamly(system_prompt, messages, settings)
        +tts(text)
        +bind_tools(session, tools)
    }
    class KnowledgebaseService {
        +get_by_ids(kb_ids)
        +get_field_map(kb_ids)
    }
    class TenantLLMService {
        +llm_id2llm_type(llm_id)
        +get_model_config(tenant_id, llm_type, llm_id)
    }
    class Langfuse {
        +auth_check()
        +create_trace_id()
        +start_generation(...)
    }
    class DeepResearcher {
        +thinking(kbinfos, question)
    }
    class MindMapExtractor {
        +__call__(contents)
        +output
    }
    DialogService ..> Dialog : model
    DialogService ..> LLMBundle : uses
    DialogService ..> KnowledgebaseService : uses
    DialogService ..> TenantLLMService : uses
    DialogService ..> Langfuse : uses
    DialogService ..> DeepResearcher : uses
    DialogService ..> MindMapExtractor : uses

    class chat {
        +chat(dialog, messages, stream=True, **kwargs)
    }
    class chat_solo {
        +chat_solo(dialog, messages, stream=True)
    }
    class ask {
        +ask(question, kb_ids, tenant_id, chat_llm_name, search_config)
    }
    class gen_mindmap {
        +gen_mindmap(question, kb_ids, tenant_id, search_config)
    }
    DialogService ..> chat
    DialogService ..> chat_solo
    DialogService ..> ask
    DialogService ..> gen_mindmap

Summary

The dialog_service.py file orchestrates dialog management, question answering, and knowledge retrieval by interfacing with LLMs, knowledge bases, and external services. It supports advanced features like streaming chat, SQL-based retrieval, citation management, metadata filtering, and mind map generation. The integration with Langfuse for tracing and the modular design with LLM bundles and retrieval services make it a central and sophisticated component of the InfiniFlow platform.

End of Documentation for dialog_service.py