test_create_chat_assistant.py

Overview

test_create_chat_assistant.py is a comprehensive test suite for validating the creation and authorization of chat assistants within the InfiniFlow platform. It uses the pytest framework to systematically verify the behavior of the create_chat_assistant function, which is responsible for creating chat assistants with specific configurations such as name, datasets, avatars, language models (LLMs), and prompts.

The tests ensure that:

• Authorization tokens are correctly validated.

• Chat assistant creation respects constraints like name uniqueness, length limits, and valid dataset ownership.

• Various configuration parameters (LLM settings, prompt structure, avatar encoding) are handled correctly.

• Error cases and edge conditions are properly managed.

This file plays a crucial role in maintaining the robustness and reliability of chat assistant creation functionality in the system.

Classes and Their Responsibilities

1. TestAuthorization

• Purpose: Tests the authorization mechanism when creating a chat assistant.

• Methods:

  test_invalid_auth

  • Parameters:

    • invalid_auth (can be None or an invalid RAGFlowHttpApiAuth instance)

    • expected_code (int): Expected response code from the API.

    • expected_message (str): Expected error message.

  • Functionality: Calls create_chat_assistant with invalid authorization tokens and verifies that the system returns the correct error codes and messages.

  • Usage Example:

    res = create_chat_assistant(None)
    assert res['code'] == 0
    assert res['message'] == "`Authorization` can't be empty"
    

  • Test Coverage: Tests for empty authorization and invalid API tokens.

2. TestChatAssistantCreate

• Purpose: Tests various aspects of chat assistant creation including validation of name, dataset ownership, avatar encoding, LLM configuration, and prompt customization.

• Fixtures: Uses clear_chat_assistants to ensure a clean state before tests.

• Methods:

  test_name

  • Parameters:

    • HttpApiAuth (valid authorization object provided by fixture)

    • add_chunks (fixture likely producing dataset IDs)

    • payload (dict): Contains the chat assistant creation payload with the name field to test various name scenarios.

    • expected_code (int): Expected response code.

    • expected_message (str): Expected error message.

  • Functionality: Validates name length limits, type, emptiness, and duplication (case-insensitive).

  • Usage Example:

    payload = {"name": "valid_name", "dataset_ids": []}
    res = create_chat_assistant(HttpApiAuth, payload)
    assert res["code"] == 0
    assert res["data"]["name"] == "valid_name"
    

  • Notes: Some test cases are skipped due to known issues.

  test_dataset_ids

  • Parameters:

    • HttpApiAuth

    • add_chunks

    • dataset_ids (can be list, callable returning list, or invalid string)

    • expected_code

    • expected_message

  • Functionality: Tests correct dataset ownership validation and handling of invalid dataset IDs.

  • Usage Example:

    payload = {"name": "ragflow test", "dataset_ids": []}
    res = create_chat_assistant(HttpApiAuth, payload)
    assert res["code"] == 0
    

  test_avatar

  • Parameters:

    • HttpApiAuth

    • tmp_path (pytest temporary directory for creating image files)

  • Functionality: Verifies that avatars are correctly encoded and accepted during chat assistant creation.

  • Usage Example:

    fn = create_image_file(tmp_path / "ragflow_test.png")
    payload = {"name": "avatar_test", "avatar": encode_avatar(fn), "dataset_ids": []}
    res = create_chat_assistant(HttpApiAuth, payload)
    assert res["code"] == 0
    

  test_llm

  • Parameters:

    • HttpApiAuth

    • add_chunks

    • llm (dict): Different configurations for the language model.

    • expected_code

    • expected_message

  • Functionality: Validates LLM parameters such as model name, temperature, top_p, penalties, and max tokens, including default fallback values.

  • Usage Example:

    payload = {"name": "llm_test", "dataset_ids": [dataset_id], "llm": {"model_name": "glm-4"}}
    res = create_chat_assistant(HttpApiAuth, payload)
    assert res["code"] == 0
    assert res["data"]["llm"]["model_name"] == "glm-4"
    

  test_prompt

  • Parameters:

    • HttpApiAuth

    • add_chunks

    • prompt (dict): Various prompt-related configurations.

    • expected_code

    • expected_message

  • Functionality: Tests prompt configuration including similarity thresholds, weights, variables, empty responses, openers, and prompt text validation. Also checks the default prompt settings when none are provided.

  • Notes: Some invalid or edge cases are skipped.

  • Usage Example:

    prompt = {"similarity_threshold": 0.5, "empty_response": "No data found."}
    payload = {"name": "prompt_test", "dataset_ids": [dataset_id], "prompt": prompt}
    res = create_chat_assistant(HttpApiAuth, payload)
    assert res["code"] == 0
    assert res["data"]["prompt"]["similarity_threshold"] == 0.5
    assert res["data"]["prompt"]["empty_response"] == "No data found."
    

3. TestChatAssistantCreate2

• Purpose: Additional test case focusing on ownership validation of parsed documents.

• Methods:

  test_unparsed_document

  • Parameters:

    • HttpApiAuth

    • add_document (fixture to add a document dataset)

  • Functionality: Tests that chat assistant creation fails when the dataset contains unparsed documents not owned by the user.

  • Usage Example:

    payload = {"name": "prompt_test", "dataset_ids": [dataset_id]}
    res = create_chat_assistant(HttpApiAuth, payload)
    assert res["code"] == 102
    assert "doesn't own parsed file" in res["message"]
    

Important Implementation Details

• Parameterization: Extensive use of pytest.mark.parametrize allows testing many edge cases and input variations systematically.

• Fixtures: The tests rely on fixtures such as HttpApiAuth, add_chunks, add_document, and clear_chat_assistants to set up necessary authorization and datasets for testing.

• Skipping Tests: Some test cases are skipped (marked with pytest.mark.skip) due to known issues or invalid inputs that are not handled yet.

• Error Codes: The system returns specific codes (e.g., 0 for success, 100/102/109 for various errors) and messages which tests verify to ensure correct API behavior.

• Avatar Handling: Tests include creating temporary image files and encoding them to verify avatar upload functionality.

• Prompt Validation: Special attention is given to prompt text, ensuring it contains required placeholders like {knowledge} and validates the types of prompt parameters.

• LLM Defaults: If no LLM configuration is supplied, the system sets default values for model name, temperature, and other parameters.

Interactions with Other Parts of the System

• create_chat_assistant function (from common): Core function tested here; responsible for API interaction to create chat assistants.

• RAGFlowHttpApiAuth (from libs.auth): Handles API authentication used in tests to simulate authorized or unauthorized requests.

• Configuration constants (CHAT_ASSISTANT_NAME_LIMIT, INVALID_API_TOKEN from configs): Used to test limits and invalid tokens.

• Utility functions (encode_avatar from utils, create_image_file from utils.file_utils): Used to handle avatar image encoding and file creation for testing.

• Fixtures (not defined here): Provide setup and teardown for datasets, documents, and authorization contexts.

• Error Handling and Codes: The tested functions return structured responses with code, message, and sometimes data fields to indicate success or specific failure reasons.

Usage and Test Workflow Summary

1. Authorization Tests: Confirm that missing or invalid API keys are rejected.

2. Name Validation: Verify that chat assistant names are valid, unique, and within length limits.

3. Dataset Ownership: Ensure users can only assign datasets they own.

4. Avatar Upload: Check that valid avatars are accepted.

5. LLM Configuration: Test various valid and invalid LLM parameter combinations.

6. Prompt Configuration: Validate prompt parameters and default behavior.

7. Document Parsing: Ensure datasets with unparsed documents are rejected.

Visual Diagram: Class and Methods Overview

classDiagram
    class TestAuthorization {
        +test_invalid_auth(invalid_auth, expected_code, expected_message)
    }

    class TestChatAssistantCreate {
        +test_name(HttpApiAuth, add_chunks, payload, expected_code, expected_message)
        +test_dataset_ids(HttpApiAuth, add_chunks, dataset_ids, expected_code, expected_message)
        +test_avatar(HttpApiAuth, tmp_path)
        +test_llm(HttpApiAuth, add_chunks, llm, expected_code, expected_message)
        +test_prompt(HttpApiAuth, add_chunks, prompt, expected_code, expected_message)
    }

    class TestChatAssistantCreate2 {
        +test_unparsed_document(HttpApiAuth, add_document)
    }

    TestAuthorization --> create_chat_assistant
    TestChatAssistantCreate --> create_chat_assistant
    TestChatAssistantCreate2 --> create_chat_assistant

Summary

The test_create_chat_assistant.py file is a critical part of the InfiniFlow testing suite focused on validating the chat assistant creation API. It ensures security (authorization), data integrity (name and dataset validation), and feature correctness (avatars, LLMs, prompts). The tests are well-structured with parameterization for broad coverage and rely on external fixtures and utility functions to simulate realistic scenarios.

This file helps maintain the quality and correctness of the chat assistant creation process, catching regressions and preventing invalid configurations from entering production.