conftest.py

Overview

The conftest.py file is a configuration and utility module used in the testing framework of the InfiniFlow project, leveraging pytest fixtures and helper functions to facilitate automated testing workflows related to document processing and chat assistant management.

This file primarily provides:

• A pytest fixture to setup and teardown chat assistants and documents for tests.

• A custom condition function to wait for document parsing completion using a retry/wait decorator.

• Integration with common helper functions for document and chat assistant lifecycle management.

The purpose of this file is to streamline test setup by automating the creation, parsing, and cleanup of test data entities, enabling reliable and repeatable tests for components that depend on documents and chat assistants.

Detailed Explanation

Imports

• pytest: The testing framework used.

• common: Module importing utility functions:

  • batch_create_chat_assistants

  • delete_chat_assistants

  • list_documents

  • parse_documents

• utils: Module importing the wait_for decorator used to retry/wait for asynchronous conditions.

Functions and Fixtures

condition(_auth, _dataset_id)

• Purpose:
  Checks whether all documents in a given dataset have completed parsing.

• Parameters:

  • _auth (object): Authentication credentials or context for API calls.

  • _dataset_id (str or int): Identifier of the dataset whose documents are being queried.

• Returns:

  • True if all documents have "run" status "DONE".

  • False otherwise.

• Implementation Details:
  This function calls list_documents to retrieve the current documents in the dataset. It iterates through each document to verify if the parsing process is complete (doc["run"] == "DONE"). If any document is still processing, it returns False.

• Decorator:
  @wait_for(30, 1, "Document parsing timeout")
  This decorator retries the function every 1 second for up to 30 seconds, raising an exception if the condition is not met within this timeout. This makes the function suitable for polling asynchronous document processing.

• Usage Example:

  if condition(auth, dataset_id):
      print("All documents parsed successfully.")
  else:
      print("Documents still parsing or failed.")
  

add_chat_assistants_func(request, HttpApiAuth, add_document)

• Type:
  Pytest fixture with function scope.

• Purpose:
  To set up chat assistants and parsed documents before a test runs and ensure cleanup afterward.

• Parameters:

  • request (pytest fixture): Provides access to the requesting test context, allowing registration of finalizers for cleanup.

  • HttpApiAuth (fixture/parameter): Authentication token or session for making API requests.

  • add_document (fixture/parameter): A tuple (dataset_id, document_id) representing a newly added document.

• Returns:

  • Tuple (dataset_id, document_id, chat_assistants) where:

    • dataset_id (str/int): Dataset identifier.

    • document_id (str/int): Document identifier within the dataset.

    • chat_assistants (list): List of 5 newly created chat assistant entities.

• Implementation Details:

  • The fixture registers a cleanup function cleanup() via request.addfinalizer that deletes all chat assistants after the test completes to maintain a clean state.

  • It triggers document parsing for the added document by calling parse_documents.

  • It waits for parsing to complete by calling the condition function, which uses the wait_for decorator to poll until all documents are done.

  • Finally, it creates 5 chat assistants using batch_create_chat_assistants and returns the relevant identifiers and created assistants.

• Usage Example (in a test function):

  def test_chat_assistant_functionality(add_chat_assistants_func):
      dataset_id, document_id, assistants = add_chat_assistants_func
      assert len(assistants) == 5
      # Further test logic here
  

Important Implementation Details

• Asynchronous Polling with wait_for Decorator:
  The condition function is decorated with wait_for, a utility that implements a retry loop with timeout and interval parameters. This pattern is critical for tests that depend on external asynchronous processes, such as document parsing that may take variable time.

• Test Resource Lifecycle Management:
  Using pytest's request.addfinalizer, the fixture ensures resources (chat assistants) are cleaned up regardless of test outcome, preventing test pollution and side effects.

• Batch Creation and Deletion of Chat Assistants:
  The use of batch_create_chat_assistants and delete_chat_assistants encapsulates bulk operations on chat assistant entities, abstracting away the lower-level API or database calls.

Interaction with Other Parts of the System

• common Module:
  Provides core operations related to documents and chat assistants, like creation, deletion, listing, and parsing. This file relies on these common utilities to perform domain-specific logic without duplicating code.

• utils Module:
  Supplies the wait_for decorator used to handle asynchronous waiting and polling.

• Test Suite:
  This conftest.py acts as a shared fixture provider within the pytest ecosystem for tests that require pre-existing documents and chat assistants, ensuring consistent setup and teardown across multiple test modules.

Visual Diagram

The file is a utility fixture module with two main functions and no classes. A flowchart best illustrates the workflow and relationships between the functions in this file.

flowchart TD
    A[Start Test using add_chat_assistants_func Fixture] --> B[add_chat_assistants_func Fixture Invoked]
    B --> C[Register cleanup() to delete chat assistants after test]
    B --> D[Retrieve dataset_id and document_id from add_document]
    D --> E[Call parse_documents to parse document]
    E --> F[Call condition() to wait for document parsing completion]
    F -->|Polling| F
    F --> G[Create 5 chat assistants with batch_create_chat_assistants]
    G --> H[Return dataset_id, document_id, chat_assistants to test]
    H --> I[Test executes with prepared data]
    I --> J[Test completes]
    J --> K[cleanup() runs to delete chat assistants]
    K --> L[End]

Summary

conftest.py is a pytest configuration file that provides automated setup and teardown of test data involving documents and chat assistants. It features a polling mechanism to wait for document parsing completion and ensures proper cleanup of chat assistants created during tests. This file plays a critical role in maintaining reliable, isolated, and repeatable test environments within the InfiniFlow testing framework.