retrieval_relevant_rewrite_and_generate.json

Overview

This JSON file defines a directed workflow graph for a retrieval-augmented generation (RAG) pipeline designed to process user queries by rewriting, retrieval, relevance checking, and answer generation stages. The pipeline begins with an initial user interaction and progresses through a series of specialized nodes, each representing a logical step or component in the information retrieval and response generation process.

The primary purpose of this file is to outline the structure, parameters, and flow of a multi-step question-answering system that leverages knowledge base retrieval, relevance assessment, question rewriting, and generative language modeling to produce accurate and context-aware answers.

Detailed Explanation of Components

Nodes

Each node represents a processing step or action in the pipeline. Nodes have the following key attributes:

• id: Unique identifier for the node.

• type: Defines the kind of node (e.g., beginNode, ragNode, relevantNode).

• position: Coordinates for UI layout (not relevant for logic).

• data: Contains node-specific settings such as labels, names, and configuration options.

• sourcePosition / targetPosition: Visual connection points for edges.

Node Types and Their Roles

1. Begin Node (begin)

• Type: beginNode

• Role: Starting point of the workflow.

• Data:

  • label: "Begin"

  • name: "FiftyDeerDeny" (internal identifier)

  • form.prologue: Initial greeting message "Hi there!"

• Functionality:

  • Initiates the flow with a prologue greeting.

  • Connects to the first answer node.

2. Answer Node (answer:0)

• Type: ragNode

• Role: Represents the initial answering step, possibly a placeholder or a step where a preliminary answer is assembled.

• Data:

  • label: "Answer"

  • name: "NinePointsSmoke"

• Functionality:

  • Receives input from both the begin node and the generate node.

  • Sends its output to the retrieval node.

3. Retrieval Node (retrieval:0)

• Type: ragNode

• Role: Retrieves relevant documents or knowledge base entries based on the input query.

• Data:

  • label: "Retrieval"

  • name: "ProudLiesPull"

  • form:

    • similarity_threshold: 0.2 — minimum similarity score for results to be considered relevant.

    • keywords_similarity_weight: 0.3 — weight given to keyword matching in similarity scoring.

    • top_n: 6 — number of top results to return.

    • top_k: 1024 — maximum number of candidates considered in retrieval.

    • rerank_id: "BAAI/bge-reranker-v2-m3" — model used for reranking retrieved documents.

    • kb_ids: List of knowledge base IDs to query.

    • empty_response: Message when no relevant information is found.

• Functionality:

  • Performs semantic search over specified knowledge base.

  • Uses reranking to refine top results.

  • Passes results downstream for relevance assessment.

4. Relevant Node (relevant:0)

• Type: relevantNode

• Role: Determines if retrieved documents are relevant enough to generate a direct answer or if the question needs rewriting.

• Data:

  • label: "Relevant"

  • name: "StrongBooksPay"

  • form:

    • llm_id: "deepseek-chat" — language model used for relevance checking.

    • temperature: 0.02 — inference temperature (low for deterministic behavior).

    • yes: "generate:0" — next node if relevant documents are found.

    • no: "rewrite:0" — next node if documents are irrelevant.

• Functionality:

  • Acts as a decision gate.

  • Routes flow to generate answer or rewrite question.

5. Generate Node (generate:0)

• Type: ragNode

• Role: Generates the final answer using a large language model (LLM) based on retrieved knowledge.

• Data:

  • label: "Generate"

  • name: "CyanBooksTell"

  • form:

    • llm_id: "deepseek-chat"

    • prompt: Template instructing the assistant to answer based on knowledge base content, with fallback messaging if content is irrelevant.

    • temperature: 0.02 — low randomness for consistent answers.

• Functionality:

  • Synthesizes answer considering chat history and retrieved documents.

  • Inserts fallback sentence if knowledge base content is irrelevant.

6. Rewrite Question Node (rewrite:0)

• Type: ragNode

• Role: Rewrites the user question to improve retrieval results.

• Data:

  • label: "RewriteQuestion"

  • name: "SourPapersMake"

  • form:

    • llm_id: "deepseek-chat"

    • temperature: 0.8 — higher randomness to generate diverse rewrites.

• Functionality:

  • Reformulates unclear or ambiguous queries.

  • After rewriting, the flow returns to the retrieval node to attempt a new search.

Edges (Workflow Connections)

Edges define transitions between nodes. The flow is:

• Begin → Answer

• Generate → Answer

• Answer → Retrieval

• Rewrite → Retrieval

• Retrieval → Relevant

• Relevant → Generate (if relevant)

• Relevant → Rewrite (if not relevant)

This cyclical design enables iterative refinement: if the retrieved content is not relevant, the question is rewritten and retrieval is attempted again.

Usage Examples

Example Scenario: User Query Processing

1. Start: The user initiates a query; the system greets with "Hi there!".

2. Answer Node: Collects initial input or context.

3. Retrieval: Searches the knowledge base for relevant documents based on the query.

4. Relevance Check: The system checks if retrieved documents are relevant.

   • If yes: Proceeds to generate a final answer.

   • If no: Rewrites the question to improve retrieval.

5. Generate Answer: Uses an LLM prompt to formulate the answer incorporating retrieved knowledge.

6. Loop: If rewriting occurs, retrieval and relevance checking repeat until a satisfactory answer is generated or the system determines no relevant content exists.

Important Implementation Details

• Semantic Search and Reranking: Retrieval uses a two-step process — initial retrieval with a high candidate number (top_k=1024), then reranking with a specialized model (BAAI/bge-reranker-v2-m3) to ensure top results are highly relevant.

• Relevance Decision Logic: Uses a low-temperature LLM to make a deterministic yes/no decision on document relevance, directing the flow accordingly.

• Question Rewriting: Employs a higher-temperature LLM invocation to generate varied and potentially clearer versions of the question, enhancing retrieval performance.

• Fallback Messaging: When the knowledge base yields no relevant information, the generated answer explicitly states this, improving user experience.

Interaction with Other System Components

• Knowledge Bases (KBs): Retrieval node queries specified KBs by ID ("869a236818b811ef91dffa163e197198"), which are external data sources containing domain-specific information.

• Large Language Models (LLMs): The system interfaces with LLMs (identified as "deepseek-chat" and reranker "BAAI/bge-reranker-v2-m3") for rewriting, relevance checking, generating answers, and reranking.

• Chat History and Context: The generate node prompt incorporates chat history and retrieved content, suggesting integration with a conversational system managing session context.

• User Interface: Positions and labels imply this graph is visualized in a UI allowing users or developers to understand and possibly modify the workflow.

Visual Diagram: Workflow Flowchart

flowchart TD
    Begin(["Begin<br/>(FiftyDeerDeny)"])
    Answer(["Answer<br/>(NinePointsSmoke)"])
    Retrieval(["Retrieval<br/>(ProudLiesPull)"])
    Relevant(["Relevant<br/>(StrongBooksPay)"])
    Generate(["Generate<br/>(CyanBooksTell)"])
    Rewrite(["Rewrite Question<br/>(SourPapersMake)"])
    
    Begin --> Answer
    Generate --> Answer
    Answer --> Retrieval
    Rewrite --> Retrieval
    Retrieval --> Relevant
    Relevant -- Yes --> Generate
    Relevant -- No --> Rewrite

Diagram Explanation:

• The flow begins at Begin, moves to Answer, then to Retrieval.

• After retrieval, Relevant decides if the documents are good enough.

• If yes, the system Generates an answer and loops back to Answer.

• If no, the question is Rewritten and retrieval is retried.

Summary

This file defines a modular, iterative RAG pipeline that orchestrates user query processing through a combination of retrieval, relevance assessment, question rewriting, and answer generation. It balances deterministic and probabilistic LLM usage with semantic search techniques to maximize the accuracy and relevance of answers derived from structured knowledge bases within a conversational context.