Test Discovery and Collection

This module implements the mechanisms by which pytest locates and collects Python test files, classes, functions, and methods based on user-configured naming conventions and patterns. It builds a hierarchical collection tree representing the test suite, enabling pytest to later execute and report on these tests.

Core Concepts and Purpose

• Test Discovery is the process of identifying which files and Python objects constitute tests.

• Collection Tree: Tests are represented as a tree of nodes, composed of collectors (for directories, packages, modules, and classes) and items (individual test functions/methods).

• Configurability: Users customize which files, classes, and functions are considered tests via ini options such as python_files, python_classes, and python_functions.

• Integration with Fixtures and Parametrization: Collected test functions are prepared for execution with fixtures and can be parameterized to generate multiple test invocations.

This module solves the problem of flexible, extensible, and accurate identification of tests in arbitrary Python codebases, respecting naming conventions, user configuration, and test organization patterns.

Key Functionalities and Workflows

1. File and Directory Collection

• Directory Collection: When pytest encounters a directory during discovery, it determines if the directory is a Python package (contains __init__.py) or a plain directory.

  • If it is a package, a Package collector node is created, which collects its contents recursively.

  • If not a package, collection may be delegated to other collectors or ignored.

• File Collection: For each file encountered, pytest checks if the filename matches configured patterns (default: test_*.py or *_test.py). Files matching these patterns are collected as Module nodes.

• Hooks for Collection: Pytest provides hooks like pytest_collect_directory and pytest_collect_file to allow plugins to customize collection behavior.

2. Module Collection

• A Module collector represents a Python test module (a single .py file).

• The module is imported dynamically during collection (importtestmodule), resolving any import errors or syntax errors and raising collection errors if needed.

• The module object is inspected for test classes and functions.

3. Class and Function Collection

• Class Collector: Collects test methods and nested classes within a class.

  • Uses filters based on configured class name patterns (python_classes).

  • Skips classes that have __init__ or new constructors to avoid unexpected instantiation.

  • Registers xunit-style setup/teardown methods (e.g., setup_class, teardown_method) as autouse fixtures.

• Function Item: Represents a single test function or method.

  • Functions are filtered by configured function name patterns (python_functions).

  • Supports parameterization, generating multiple Function items for different parameter sets.

  • Hooks like pytest_pyfunc_call are used to execute the underlying test function.

4. Parametrization During Collection

• The Metafunc object is created during collection for each function, allowing hooks to add parametrized invocations.

• Parametrization can be done via the @pytest.mark.parametrize decorator or programmatically.

• Each parameter set corresponds to a distinct Function item with unique node IDs.

• Unique IDs for parameterized tests are generated to avoid collisions, supporting user-defined IDs, hooks, and automatic generation.

5. Test Identification Filters

• The module implements filters for determining if an object is a test class or function, based on:

  • Matching configured prefixes or glob patterns.

  • Presence of test attribute (e.g., for nose compatibility).

  • Ensuring the object is callable and not marked as a fixture.

• These filters prevent accidental collection of non-test code.

6. Setup and Teardown Integration

• Classic xunit-style setup and teardown functions (setup_module, teardown_function, setup_method, etc.) are integrated into the fixture system as autouse fixtures to unify setup behavior.

Interaction with Other System Components

• Nodes Module (_pytest.nodes): Defines the base classes for nodes in the collection tree (Node, Collector, Item). The Python collection classes (Module, Class, Function) inherit from these base classes.

• Fixtures Module (_pytest.fixtures): Collection integrates with the fixture system by parsing fixture factories and registering setup/teardown behavior.

• Config Module (_pytest.config): Provides configuration options and ini values that control discovery patterns.

• Hooks: Various collection-related hooks allow extension points for plugins to customize discovery.

• Session: The collection tree nodes are linked to the test session, which manages the overall test run lifecycle.

Important Concepts and Design Patterns

• Node Tree Construction: The collection process constructs a tree of nodes representing directories, packages, files, classes, and functions. Each node knows its parent and children.

• Indirection in Construction: Nodes use a from_parent factory method for construction rather than direct constructors, improving flexibility and supporting future refactoring.

• Use of Python Introspection: Collection uses Python's inspection capabilities (inspect module) to detect classes, functions, and their properties.

• Integration with Parametrization and Fixtures: Collection is tightly coupled with test parametrization and fixture resolution, preparing test items for execution.

• Error Handling: Collection wraps import and introspection errors into Collector.CollectError to report issues during discovery without crashing pytest.

• Hooks and Plugin Extensibility: Collection is designed with hooks to allow external plugins to customize or override default behaviors.

Illustrative Code References

Directory Collection Decision

def pytest_collect_directory(path: Path, parent: nodes.Collector) -> nodes.Collector | None:
    pkginit = path / "__init__.py"
    try:
        has_pkginit = pkginit.is_file()
    except PermissionError:
        return None
    if has_pkginit:
        return Package.from_parent(parent, path=path)
    return None

This logic decides whether a directory is a package collector or not.

File Collection Filtering

def pytest_collect_file(file_path: Path, parent: nodes.Collector) -> nodes.Module | None:
    if file_path.suffix == ".py":
        if not parent.session.isinitpath(file_path):
            if not path_matches_patterns(file_path, parent.config.getini("python_files")):
                return None
        ihook = parent.session.gethookproxy(file_path)
        module = ihook.pytest_pycollect_makemodule(module_path=file_path, parent=parent)
        return module
    return None

Files not matching configured test file patterns are ignored.

Class Filtering for Test Classes

def istestclass(self, obj: object, name: str) -> bool:
    if not (self.classnamefilter(name) or self.isnosetest(obj)):
        return False
    if inspect.isabstract(obj):
        return False
    return True

Classes are filtered by name and other checks before being collected.

Function Filtering for Test Functions

def istestfunction(self, obj: object, name: str) -> bool:
    if self.funcnamefilter(name) or self.isnosetest(obj):
        if isinstance(obj, (staticmethod, classmethod)):
            obj = safe_getattr(obj, "__func__", False)
        return callable(obj) and fixtures.getfixturemarker(obj) is None
    else:
        return False

Only functions matching naming conventions and not marked as fixtures are collected as tests.

Parametrization Processing

def pytest_generate_tests(metafunc: Metafunc) -> None:
    for marker in metafunc.definition.iter_markers(name="parametrize"):
        metafunc.parametrize(*marker.args, **marker.kwargs, _param_mark=marker)

Hooks allow `@pytest.mark.parametrize` to generate multiple test invocations at collection time.

Mermaid Diagram: Test Discovery and Collection Workflow

flowchart TD
    Start[Start Test Discovery]
    DirCheck{Is Directory?}
    HasInitPy[Contains __init__.py?]
    PackageCollector[Create Package Collector]
    PlainDirCollector[Create Plain Directory Collector]
    FileCheck{Is Python File (*.py)?}
    MatchesFilePattern{Matches python_files pattern?}
    ModuleCollector[Create Module Collector]
    ImportModule[Import Module]
    CollectClasses[Collect Test Classes]
    CollectFunctions[Collect Test Functions]
    Parametrize[Process Parametrization]
    CollectSetupTeardown[Register Setup/Teardown Fixtures]
    End[Collection Complete]

    Start --> DirCheck
    DirCheck -- Yes --> HasInitPy
    HasInitPy -- Yes --> PackageCollector --> PackageCollector --> End
    HasInitPy -- No --> PlainDirCollector --> End
    DirCheck -- No --> FileCheck
    FileCheck -- Yes --> MatchesFilePattern
    MatchesFilePattern -- Yes --> ModuleCollector --> ImportModule --> CollectClasses --> CollectFunctions --> Parametrize --> CollectSetupTeardown --> End
    MatchesFilePattern -- No --> End
    FileCheck -- No --> End

This flowchart depicts how pytest discovers and collects tests starting from the filesystem, through directories and files, importing modules, and collecting classes and functions, including handling parametrization and setup registration.

Summary

The **Test Discovery and Collection** module is responsible for translating a user’s test directory and file structure into an internal collection tree of test nodes. It applies naming conventions and user configurations to identify test modules, classes, and functions, supports parametrized tests, and integrates setup/teardown methods as fixtures. This process lays the foundation for pytest's test execution and reporting capabilities.