test_create_chat_assistant.py

Overview

This file contains a comprehensive suite of automated tests for the chat assistant creation functionality in the InfiniFlow project. Using the pytest framework, these tests validate the behavior and robustness of the create_chat_assistant API endpoint, ensuring that chat assistants are created correctly according to various input parameters and constraints.

The tests cover:

• Authorization validation for API requests.

• Validation of chat assistant attributes such as name, dataset ownership, avatar images, language model (LLM) parameters, and prompt configurations.

• Handling of edge cases and invalid inputs.

• Ensuring proper error codes and messages are returned.

• Interaction with dataset and document ownership.

This file plays a critical role in the continuous integration pipeline by preventing regressions in the chat assistant creation process.

Dependencies and Imports

• pytest — testing framework used to implement parametric and fixture-based tests.

• create_chat_assistant — function under test, responsible for creating chat assistants.

• RAGFlowHttpApiAuth — authentication class for API requests.

• Utility methods for encoding avatars (encode_avatar) and creating temporary image files (create_image_file).

• Various constants such as CHAT_ASSISTANT_NAME_LIMIT and INVALID_API_TOKEN used for validation.

Test Classes and Methods

1. TestAuthorization

Tests the authorization mechanism for creating chat assistants.

Method: test_invalid_auth

• Parameters:

  • auth: Authentication object or None.

  • expected_code: Expected error code integer.

  • expected_message: Expected error message string.

• Behavior:
  Calls create_chat_assistant with invalid or missing authentication and asserts that the response contains the correct error code and message.

• Usage Example:

auth = RAGFlowHttpApiAuth(INVALID_API_TOKEN)
response = create_chat_assistant(auth)
assert response["code"] == 109
assert "API key is invalid" in response["message"]

2. TestChatAssistantCreate

Tests various aspects of the chat assistant creation process, including name validation, dataset ownership, avatar handling, LLM settings, and prompt parameters.

This class uses the clear_chat_assistants fixture to ensure a clean state before each test.

Method: test_name

• Parameters:

  • payload: Dictionary containing at least the name field.

  • expected_code: Expected response code.

  • expected_message: Expected error message if applicable.

• Functionality:
  Tests valid and invalid assistant names, including empty names, names exceeding length limits, non-string names, and duplicate names (case insensitive).

• Important Notes:
  Some test cases are skipped due to known issues.

• Example:

payload = {"name": "valid_name", "dataset_ids": []}
res = create_chat_assistant(auth, payload)
assert res["code"] == 0
assert res["data"]["name"] == "valid_name"

Method: test_dataset_ids

• Parameters:

  • dataset_ids: List of dataset IDs or a callable returning a list.

  • expected_code: Expected response code.

  • expected_message: Expected error message if applicable.

• Functionality:
  Validates that only datasets owned by the user can be associated with the chat assistant. Tests empty, valid, and invalid dataset IDs.

• Example:

payload = {"name": "assistant", "dataset_ids": ["invalid_dataset_id"]}
res = create_chat_assistant(auth, payload)
assert res["code"] == 102
assert "don't own the dataset" in res["message"]

Method: test_avatar

• Parameters:

  • tmp_path: Temporary directory fixture for file creation.

• Functionality:
  Tests uploading and encoding of avatar images for chat assistants.

• Example:

fn = create_image_file(tmp_path / "avatar.png")
payload = {"name": "avatar_test", "avatar": encode_avatar(fn), "dataset_ids": []}
res = create_chat_assistant(auth, payload)
assert res["code"] == 0

Method: test_llm

• Parameters:

  • llm: Dictionary with LLM configuration parameters.

  • expected_code: Expected response code.

  • expected_message: Expected message for errors.

• Functionality:
  Tests various valid and invalid configurations of the language model parameters such as model_name, temperature, top_p, presence_penalty, frequency_penalty, and max_token.

• Behavior:
  Verifies that default LLM parameters are set if none are provided, and that invalid values yield appropriate error messages or are skipped.

• Example:

llm_config = {"model_name": "glm-4"}
payload = {"name": "llm_test", "dataset_ids": [dataset_id], "llm": llm_config}
res = create_chat_assistant(auth, payload)
assert res["code"] == 0
assert res["data"]["llm"]["model_name"] == "glm-4"

Method: test_prompt

• Parameters:

  • prompt: Dictionary of prompt-related parameters.

  • expected_code: Expected response code.

  • expected_message: Expected error message.

• Functionality:
  Validates prompt configuration options such as similarity thresholds, weights, variables, empty response messages, openers, and whether to show quotes. Also checks for usage of variables in the prompt string.

• Important Details:

  • The prompt string must include the {knowledge} variable unless variables are explicitly empty.

  • Some test cases are skipped due to known issues.

• Example:

prompt_config = {"prompt": "Hello World {knowledge}", "variables": []}
payload = {"name": "prompt_test", "dataset_ids": [dataset_id], "prompt": prompt_config}
res = create_chat_assistant(auth, payload)
assert res["code"] == 0
assert "knowledge" in res["data"]["prompt"]["prompt"]

3. TestChatAssistantCreate2

Additional tests focusing on document ownership validation in chat assistant creation.

Method: test_unparsed_document

• Parameters:

  • get_http_api_auth: Authentication fixture.

  • add_document: Fixture providing a dataset ID and document ID.

• Functionality:
  Tests that chat assistants cannot be created with datasets containing unparsed documents, verifying error code 102 and appropriate error message.

• Example:

payload = {"name": "prompt_test", "dataset_ids": [dataset_id]}
res = create_chat_assistant(auth, payload)
assert res["code"] == 102
assert "doesn't own parsed file" in res["message"]

Important Implementation Details

• Parameter Validation:
  The tests cover rigorous validation of input parameters to ensure the API's robustness. This includes type checking, value range enforcement, and business logic constraints (e.g., dataset ownership, uniqueness of names).

• Use of Pytest Features:

  • parametrize is extensively used to test multiple input scenarios efficiently.

  • Markers such as @pytest.mark.p1, @pytest.mark.p2, and @pytest.mark.p3 indicate priority or categorization of tests.

  • Some tests are skipped with reasons (marks=pytest.mark.skip), indicating awareness of known issues.

• Fixtures Integration:
  The tests rely on fixtures like get_http_api_auth, add_chunks, add_document, and clear_chat_assistants to provide test data and setup/teardown mechanisms.

• Response Structure:
  The tested API returns a JSON-like dictionary with at least the keys:

  • "code": Integer status code (0 for success, others for errors).

  • "message": Error or success message.

  • "data": Payload containing created chat assistant details.

Interaction with Other System Components

• create_chat_assistant API:
  This file tests the core API responsible for creating chat assistants. It is a critical integration point between the frontend/user inputs and backend services.

• Authentication (RAGFlowHttpApiAuth):
  Validates API tokens used to authorize requests.

• Datasets and Documents Management:
  Tests ensure chat assistants can only be created with datasets and documents owned and properly parsed by the user, enforcing data security and consistency.

• Utility Functions:
  Uses utilities for encoding avatars and creating image files to simulate user uploads.

• Constants:
  Enforces limits and error tokens defined elsewhere (CHAT_ASSISTANT_NAME_LIMIT, INVALID_API_TOKEN).

Diagram: Class Structure and Main Methods

classDiagram
    class TestAuthorization {
        +test_invalid_auth(auth, expected_code, expected_message)
    }

    class TestChatAssistantCreate {
        +test_name(payload, expected_code, expected_message)
        +test_dataset_ids(dataset_ids, expected_code, expected_message)
        +test_avatar(get_http_api_auth, tmp_path)
        +test_llm(llm, expected_code, expected_message)
        +test_prompt(prompt, expected_code, expected_message)
    }

    class TestChatAssistantCreate2 {
        +test_unparsed_document(get_http_api_auth, add_document)
    }

Summary

The test_create_chat_assistant.py file is a critical quality assurance resource for the InfiniFlow project, ensuring that chat assistants are created reliably, securely, and correctly with valid parameters and proper authorization. It uses pytest capabilities to cover a wide spectrum of input scenarios, edge cases, and error handling, thereby supporting the robustness and maintainability of the chat assistant creation process in the system.