test_create_dataset.py

Overview

test_create_dataset.py is a comprehensive test suite designed to validate the dataset creation functionality of the InfiniFlow system through its HTTP API. Utilizing the pytest framework combined with property-based testing from hypothesis, this file rigorously tests various aspects of dataset creation including authorization, input validation, concurrency, and detailed field-specific constraints.

The primary function under test is create_dataset, which interacts with the API to create datasets with different attributes. This test suite ensures that the API behaves correctly for valid inputs and gracefully handles invalid inputs, enforcing business rules and data integrity.

Detailed Explanation

Imports and Dependencies

• concurrent.futures.ThreadPoolExecutor: For testing concurrent dataset creation.

• pytest: The testing framework used.

• hypothesis: Provides property-based testing with strategies and settings.

• common: Contains constants like DATASET_NAME_LIMIT and INVALID_API_TOKEN.

• create_dataset: The main API wrapper function to create datasets.

• libs.auth.RAGFlowHttpApiAuth: Handles authentication tokens.

• libs.utils.encode_avatar: Encodes images to Base64 strings for avatars.

• libs.utils.file_utils.create_image_file: Utility to create a temporary image file for testing.

• libs.utils.hypothesis_utils.valid_names: Hypothesis strategy generating valid dataset names.

Classes and Test Cases

1. TestAuthorization

Tests related to API authorization when creating a dataset.

• test_auth_invalid

  • Parameters:

    • auth: Authorization object or None.

    • expected_code: Expected response error code.

    • expected_message: Expected error message.

  • Behavior: Tests empty or invalid API tokens.

  • Example:

    res = create_dataset(None, {"name": "auth_test"})
    assert res["code"] == 0  # Expected failure with code 0 for empty auth
    

2. TestRquest (likely a typo, should be TestRequest)

Tests API requests with invalid content types and malformed JSON payloads.

• test_content_type_bad

  • Tests sending a request with unsupported content type (text/xml).

• test_payload_bad

  • Tests malformed JSON syntax and invalid payload types (e.g., string instead of object).

3. TestCapability

Tests system capacity and concurrency.

• test_create_dataset_1k

  • Creates 1,000 datasets sequentially to test system limits.

• test_create_dataset_concurrent

  • Creates 100 datasets concurrently with a thread pool of 5 workers.

4. TestDatasetCreate

Extensive tests on dataset creation validating different fields and constraints.

• Field: name

  • Valid names tested via property-based testing.

  • Invalid cases tested with empty strings, spaces, too long names, and non-string inputs.

  • Duplicate and case-insensitive duplicates tested.

• Field: avatar

  • Tests base64-encoded image avatars.

  • Checks size limits and MIME prefix correctness.

  • Tests unset and None avatar values.

• Field: description

  • Tests valid descriptions and length limits.

  • Tests unset and None values.

• Field: embedding_model

  • Tests valid embedding models from various providers.

  • Tests invalid models, malformed formats, unset, and None.

• Field: permission

  • Validates permission values (me or team), case-insensitive and stripped.

  • Tests invalid and unset cases.

• Field: chunk_method

  • Tests various allowed chunking methods.

  • Invalid, unset, and None tested.

• Field: pagerank

  • Tests integer values within [0, 100].

  • Tests invalid values below 0, above 100, unset, and None.

• Field: parser_config

  • Tests complex nested configurations with various valid values.

  • Tests invalid values with detailed error messages.

  • Tests empty, unset, and None parser_config.

• Unsupported fields

  • Tests that extraneous fields in payloads are rejected with error.

Important Implementation Details and Algorithms

• Property-based Testing: Uses hypothesis to generate a variety of valid dataset names to test name validation comprehensively.

• Parameterized Testing: Uses pytest.mark.parametrize extensively to test multiple input variations and their expected outcomes in a single test function.

• Concurrency Testing: Uses ThreadPoolExecutor to simulate concurrent dataset creation to verify thread safety and system scalability.

• Validation Feedback: Tests confirm that error messages are clear, specific, and that error codes align with expected failure modes.

• Avatar Encoding and MIME Validation: Validates that avatar images are properly base64-encoded with correct MIME prefixes and size limits.

• Parser Config Validation: Deep validation of nested parser configuration JSON objects, ensuring each field meets expected types, ranges, and constraints.

Interactions with Other Parts of the System

• create_dataset function: The core API call tested here, likely implemented elsewhere in the system, which sends HTTP requests to the backend.

• Authentication (RAGFlowHttpApiAuth): Provides API token management and authentication validation.

• Utility modules (libs.utils): Provides helper functions for encoding images and creating temp files for avatar testing.

• Constants and fixtures: Uses constants like DATASET_NAME_LIMIT from common and pytest fixtures like clear_datasets and get_http_api_auth to set up test prerequisites.

• Hypothesis utilities: Uses custom strategies for generating valid dataset names.

This file acts as a critical quality gate ensuring the dataset creation endpoint behaves correctly, enforcing API contract and business rules.

Usage Examples

Example of a simple test case usage inside this file:

@pytest.mark.p1
@given(name=valid_names())
@example("a" * 128)
@settings(max_examples=20)
def test_name(self, get_http_api_auth, name):
    res = create_dataset(get_http_api_auth, {"name": name})
    assert res["code"] == 0
    assert res["data"]["name"] == name

This test uses a property-based approach generating various valid names to verify that dataset creation succeeds with those names.

Mermaid Diagram: Test Class Structure

classDiagram
    class TestAuthorization {
        +test_auth_invalid(auth, expected_code, expected_message)
    }
    class TestRquest {
        +test_content_type_bad(get_http_api_auth)
        +test_payload_bad(get_http_api_auth, payload, expected_message)
    }
    class TestCapability {
        +test_create_dataset_1k(get_http_api_auth)
        +test_create_dataset_concurrent(get_http_api_auth)
    }
    class TestDatasetCreate {
        +test_name(get_http_api_auth, name)
        +test_name_invalid(get_http_api_auth, name, expected_message)
        +test_name_duplicated(get_http_api_auth)
        +test_name_case_insensitive(get_http_api_auth)
        +test_avatar(get_http_api_auth, tmp_path)
        +test_avatar_exceeds_limit_length(get_http_api_auth)
        +test_avatar_invalid_prefix(get_http_api_auth, tmp_path, name, prefix, expected_message)
        +test_avatar_unset(get_http_api_auth)
        +test_avatar_none(get_http_api_auth)
        +test_description(get_http_api_auth)
        +test_description_exceeds_limit_length(get_http_api_auth)
        +test_description_unset(get_http_api_auth)
        +test_description_none(get_http_api_auth)
        +test_embedding_model(get_http_api_auth, name, embedding_model)
        +test_embedding_model_invalid(get_http_api_auth, name, embedding_model)
        +test_embedding_model_format(get_http_api_auth, name, embedding_model)
        +test_embedding_model_unset(get_http_api_auth)
        +test_embedding_model_none(get_http_api_auth)
        +test_permission(get_http_api_auth, name, permission)
        +test_permission_invalid(get_http_api_auth, name, permission)
        +test_permission_unset(get_http_api_auth)
        +test_permission_none(get_http_api_auth)
        +test_chunk_method(get_http_api_auth, name, chunk_method)
        +test_chunk_method_invalid(get_http_api_auth, name, chunk_method)
        +test_chunk_method_unset(get_http_api_auth)
        +test_chunk_method_none(get_http_api_auth)
        +test_pagerank(get_http_api_auth, name, pagerank)
        +test_pagerank_invalid(get_http_api_auth, name, pagerank, expected_message)
        +test_pagerank_unset(get_http_api_auth)
        +test_pagerank_none(get_http_api_auth)
        +test_parser_config(get_http_api_auth, name, parser_config)
        +test_parser_config_invalid(get_http_api_auth, name, parser_config, expected_message)
        +test_parser_config_empty(get_http_api_auth)
        +test_parser_config_unset(get_http_api_auth)
        +test_parser_config_none(get_http_api_auth)
        +test_unsupported_field(get_http_api_auth, payload)
    }

    TestAuthorization <|-- TestRquest

Summary

• This file is a test suite validating the dataset creation API.

• It covers authorization, request format, concurrency, and field-level validation.

• Utilizes pytest and hypothesis for expressive and thorough testing.

• Ensures API robustness and data integrity for InfiniFlow's dataset creation feature.

• Tests granular error handling and edge cases extensively.

This documentation should help developers understand the purpose, scope, and detailed functionality of the test_create_dataset.py file, supporting maintenance, extension, and debugging efforts.