conftest.py

Overview

The conftest.py file is a configuration and fixture definition module used by the pytest testing framework within the InfiniFlow project. Its primary purpose is to define reusable test fixtures that manage the lifecycle of document data in the system, including setup and teardown operations. Specifically, it provides fixtures that create and upload documents to datasets before tests run and clean up (delete) those documents afterward to ensure test isolation and prevent side effects.

This file focuses on preparing datasets with uploaded documents for testing HTTP API endpoints related to document management. It integrates with external helper functions (bulk_upload_documents and delete_documents) from a common module to perform the actual upload and deletion operations.

Detailed Descriptions

Imports

• pytest: The Python testing framework used to define fixtures and test cases.

• bulk_upload_documents (from common): Utility function to upload multiple documents to a given dataset.

• delete_documents (from common): Utility function to delete documents from a given dataset.

Fixtures

All fixtures use the pytest.fixture decorator and are designed to be used in test functions or test classes. They automate setup and teardown by uploading documents to datasets before tests run and deleting those documents after tests complete.

1. add_document_func

@pytest.fixture(scope="function")
def add_document_func(request, HttpApiAuth, add_dataset, ragflow_tmp_dir):
    ...

• Scope: Function-level (runs setup and teardown for each test function using this fixture)

• Purpose: Uploads a single document to a newly created dataset and ensures cleanup after the test.

• Parameters:

  • request: Pytest built-in fixture providing information about the requesting test context.

  • HttpApiAuth: Authentication fixture to authorize API calls.

  • add_dataset: Fixture that creates and returns a new dataset ID.

  • ragflow_tmp_dir: Temporary directory path used for document upload (likely contains test files).

• Returns: A tuple (dataset_id, document) where:

  • dataset_id is the ID of the created dataset.

  • document is the first uploaded document (returned by bulk_upload_documents).

• Usage Example:

def test_single_document(add_document_func):
    dataset_id, document = add_document_func
    # Use dataset_id and document in test assertions

• Implementation Details:

  • Registers a finalizer to delete all documents in the dataset after the test completes by calling delete_documents with {"ids": None} (indicating all documents).

  • Uploads exactly one document (count=1) to the dataset using bulk_upload_documents.

2. add_documents

@pytest.fixture(scope="class")
def add_documents(request, HttpApiAuth, add_dataset, ragflow_tmp_dir):
    ...

• Scope: Class-level (runs once per test class using this fixture)

• Purpose: Uploads five documents to a newly created dataset and ensures cleanup after all tests in the class complete.

• Parameters: Same as add_document_func.

• Returns: A tuple (dataset_id, documents) where:

  • dataset_id is the ID of the created dataset.

  • documents is a list of the five uploaded documents.

• Usage Example:

@pytest.mark.usefixtures("add_documents")
class TestDocumentBatch:
    def test_documents_count(self, add_documents):
        dataset_id, documents = add_documents
        assert len(documents) == 5

• Implementation Details:

  • Finalizer deletes all documents from the dataset after all tests in the class complete.

  • Uploads five documents (count=5) to the dataset.

3. add_documents_func

@pytest.fixture(scope="function")
def add_documents_func(request, HttpApiAuth, add_dataset_func, ragflow_tmp_dir):
    ...

• Scope: Function-level

• Purpose: Uploads three documents to a dataset created by the add_dataset_func fixture, with cleanup after each test function.

• Parameters:

  • Similar to above, except uses add_dataset_func instead of add_dataset (presumably a function-level dataset fixture).

• Returns: A tuple (dataset_id, documents) with three uploaded documents.

• Usage Example:

def test_documents_set(add_documents_func):
    dataset_id, documents = add_documents_func
    assert len(documents) == 3

• Implementation Details:

  • Same cleanup strategy as above.

  • Uploads three documents (count=3).

Important Implementation Details

• Cleanup Strategy: Each fixture registers a cleanup function as a finalizer with request.addfinalizer(cleanup). This ensures documents uploaded for the test are deleted afterward, maintaining test isolation.

• Document Upload: The fixtures rely on bulk_upload_documents to perform document uploads; this function returns uploaded document metadata, allowing tests to reference the actual documents.

• Dataset Handling: Datasets are dynamically created using other fixtures (add_dataset or add_dataset_func), allowing document upload fixtures to be agnostic of dataset creation logic.

• Scope Differences: Function-scoped fixtures run setup/teardown for each test function, while class-scoped fixtures run once per test class. This enables flexibility in test design and performance optimization.

Interaction with Other Parts of the System

• common Module: Provides core utility functions bulk_upload_documents and delete_documents, which handle API interactions or database operations for document management.

• Authentication: The HttpApiAuth fixture provides authenticated API access, required for uploading and deleting documents.

• Dataset Fixtures: add_dataset and add_dataset_func are responsible for creating datasets; this file assumes their presence and uses them to scope the document uploads.

• Tests Using These Fixtures: Test cases that require pre-populated datasets with documents will use these fixtures to automate setup and cleanup.

Visual Diagram

The following Mermaid flowchart illustrates the relationships and workflow between the fixtures and key functions in this file:

flowchart TD
    subgraph Fixtures
        A[add_document_func (function scope)]
        B[add_documents (class scope)]
        C[add_documents_func (function scope)]
    end

    subgraph Dataset Creation
        D[add_dataset]
        E[add_dataset_func]
    end

    subgraph Document Management
        F[bulk_upload_documents]
        G[delete_documents]
    end

    A --> D
    B --> D
    C --> E

    A --> F
    B --> F
    C --> F

    A --> G
    B --> G
    C --> G

    style A fill:#f9f,stroke:#333,stroke-width:1px
    style B fill:#f9f,stroke:#333,stroke-width:1px
    style C fill:#f9f,stroke:#333,stroke-width:1px

• Fixtures depend on dataset creation fixtures (add_dataset / add_dataset_func).

• Fixtures call bulk_upload_documents to upload documents before tests.

• Fixtures register finalizers that call delete_documents to clean up after tests.

Summary

conftest.py defines critical pytest fixtures that prepare datasets with documents for testing the InfiniFlow system. It automates uploading documents and cleaning them up to ensure reliable, isolated, and repeatable tests. The file leverages existing dataset fixtures and common utilities for upload and deletion, abstracting test setup complexity and promoting reuse across the test suite.