python.py

Overview

The [python.py](/projects/286/67218) module is a core part of the **pytest** testing framework responsible for **discovering**, **collecting**, and **preparing Python test functions, classes, modules, and packages** for execution. It implements the logic by which pytest:

• Finds Python files and directories that contain tests.

• Imports test modules safely and handles import errors.

• Collects test classes and functions based on user-configured naming patterns.

• Supports parametrization of tests to create multiple test invocations from a single function.

• Integrates classic xunit-style setup/teardown methods into the pytest fixture system.

• Provides hooks to allow plugins to customize or extend test collection.

This module builds a hierarchical **collection tree** of nodes representing test suites, classes, and individual test functions. This tree is later used for running the tests and generating reports.

Detailed Documentation

Functions

pytest_addoption(parser: Parser) -> None

Registers command-line and ini-file options to configure Python test discovery related to:

• python_files: Glob patterns for test module files (default ["test_*.py", "*_test.py"]).

• python_classes: Prefixes or glob names for test class discovery (default ["Test"]).

• python_functions: Prefixes or glob names for test function/method discovery (default ["test"]).

• disable_test_id_escaping_and_forfeit_all_rights_to_community_support: Boolean for disabling string escape of non-ASCII chars in test IDs.

**Usage Example:**

def pytest_addoption(parser):
    parser.addini("python_files", type="args", default=["test_*.py", "*_test.py"])
    # ... other options

pytest_generate_tests(metafunc: Metafunc) -> None

Hook implementation that enables test parametrization by applying all `@pytest.mark.parametrize` markers found on the test function. Calls `metafunc.parametrize()` accordingly to generate multiple test calls.

pytest_configure(config: Config) -> None

Adds documentation lines for the built-in markers `parametrize` and `usefixtures` for users when running `pytest --markers`.

async_fail(nodeid: str) -> None

Utility function that raises a failure with a message indicating that native async `async def` test functions are not supported without a suitable pytest plugin.

pytest_pyfunc_call(pyfuncitem: Function) -> object | None

Hook for calling test functions:

• Checks if the test function is async; if so, calls async_fail.

• Calls the test function with its fixture arguments.

• Warns if a test function returns a non-None value (pytest expects tests to not return).

• Returns True to indicate that the test was called.

pytest_collect_directory(path: Path, parent: nodes.Collector) -> nodes.Collector | None

Checks if a directory contains `__init__.py` (making it a package). If so, returns a `Package` collector node for it; otherwise, returns `None`.

pytest_collect_file(file_path: Path, parent: nodes.Collector) -> Module | None

Checks if a file is a Python file (`*.py`) and matches the configured `python_files` pattern. If so, returns a `Module` collector node for it; otherwise, returns `None`.

path_matches_patterns(path: Path, patterns: Iterable[str]) -> bool

Utility function that returns `True` if the given path matches any of the provided glob-style patterns.

pytest_pycollect_makemodule(module_path: Path, parent) -> Module

Factory function to create a `Module` collector node from a path.

pytest_pycollect_makeitem(collector: Module | Class, name: str, obj: object) -> None | nodes.Item | nodes.Collector | list[nodes.Item | nodes.Collector]

Hook implementation that attempts to create a collector or item for a given object inside a module or class:

• If object is a class and matches test class criteria, returns a Class collector.

• If object is a function and matches test function criteria, returns a list of Function items (handling generators and parametrization).

• Otherwise returns None.

importtestmodule(path: Path, config: Config)

Imports a Python module from the given path using the specified import mode. Handles import errors and wraps them as pytest collection errors. On success, registers the module with the plugin manager.

_call_with_optional_argument(func, arg) -> None

Calls the given function with the argument if the function accepts one argument; otherwise calls it without arguments. Used to invoke legacy setup/teardown functions that may or may not accept a parameter.

_get_first_non_fixture_func(obj: object, names: Iterable[str]) -> object | None

Returns the first attribute of `obj` among the given `names` that is not marked as a pytest fixture. Used to identify xunit-style setup/teardown functions.

hasinit(obj: object) -> bool

Returns `True` if the object has a custom `__init__` constructor (not the default `object.__init__`).

hasnew(obj: object) -> bool

Returns `True` if the object has a custom `__new__` constructor (not the default `object.__new__`).

_find_parametrized_scope(argnames: Sequence[str], arg2fixturedefs: Mapping[str, Sequence[FixtureDef]], indirect: bool | Sequence[str]) -> Scope

Determines the narrowest fixture scope for the parametrized arguments. Returns `Scope.Function` if any argument is direct; otherwise returns the narrowest scope of indirect fixtures.

_ascii_escaped_by_config(val: str | bytes, config: Config | None) -> str

Escapes non-ASCII characters in a string or bytes value according to the config option `disable_test_id_escaping_and_forfeit_all_rights_to_community_support`.

get_direct_param_fixture_func(request: FixtureRequest) -> Any

Fixture function used for direct parametrization that returns `request.param`.

Classes

PyobjMixin(nodes.Node)

Mixin class that extends a pytest Node to provide accessors for the underlying Python objects:

• .module: Python module object where this node was collected.

• .cls: Python class object where this node was collected.

• .instance: Python instance the test function is bound to (None for standalone functions).

• .obj: The underlying Python object represented by this node.

It implements caching of `.obj` and marker extraction.

PyCollector(PyobjMixin, nodes.Collector, abc.ABC)

Abstract base class for Python test collectors (modules, classes) that collect test functions or nested collectors.

Key methods:

• funcnamefilter(name: str) -> bool: Check if a function name matches configured test function patterns.

• classnamefilter(name: str) -> bool: Check if a class name matches configured test class patterns.

• istestfunction(obj: object, name: str) -> bool: Determine if an object is a test function.

• istestclass(obj: object, name: str) -> bool: Determine if an object is a test class.

• collect() -> Iterable[nodes.Item | nodes.Collector]: Collect child nodes (test functions, classes).

• _genfunctions(name: str, funcobj): Generates Function items for a potentially parametrized test function.

Module(nodes.File, PyCollector)

Collector for test classes and functions within a Python test module.

Key details:

• Imports the test module via importtestmodule.

• Registers autouse fixtures to integrate legacy setUpModule and tearDownModule functions.

• Registers autouse fixtures for setup_function and teardown_function.

• Calls fixture manager to parse fixtures for the module object.

Package(nodes.Directory)

Collector for Python packages (directories containing `__init__.py`).

Key details:

• On setup, imports the __init__.py module.

• Calls legacy module-level setup/teardown functions if available.

• Collects subdirectories and files, yielding collectors or items accordingly.

• Sorts __init__.py first.

Class(PyCollector)

Collector for test methods and nested classes within a Python test class.

Key details:

• Checks for presence of __init__ or __new__ and warns if present (cannot collect).

• Registers autouse fixtures for setup_class/teardown_class and setup_method/teardown_method.

• Calls fixture manager to parse fixtures for a new instance of the class.

• Collects test methods.

Function(PyobjMixin, nodes.Item)

Represents a single Python test function or method.

Key details:

• Supports parametrization via callspec.

• Caches the underlying function object and instance.

• Handles setup by filling fixtures.

• Runs the test function via the pytest hook.

• Filters tracebacks to remove pytest internals.

• Warns if test function returns non-None.

• Supports construction via from_parent.

FunctionDefinition(Function)

Placeholder subclass representing a function definition node that cannot be run as a test.

IdMaker

Helper class to generate unique IDs for parameterized tests.

Key methods:

• make_unique_parameterset_ids(): Generates unique string IDs for parameter sets.

• Supports user-provided ID functions, explicit IDs, and automatic ID generation from values.

• Ensures IDs are unique, suffixing duplicates as needed.

• Raises errors for invalid or multiple hidden IDs.

CallSpec2

Represents a single parameterized invocation of a test function.

Attributes:

• params: Mapping of argument names to values.

• indices: Argument name to index mapping.

• _arg2scope: Argument name to fixture scope.

• _idlist: List of ID tokens forming a unique test ID.

• marks: List of pytest marks applied to this callspec.

Provides method `setmulti()` to create a new callspec with additional parameters.

Metafunc

Passed to the `pytest_generate_tests` hook to allow dynamic test parametrization during collection.

Key methods:

• parametrize(argnames, argvalues, indirect=False, ids=None, scope=None, _param_mark=None)

  Adds parameter sets to the test function, supporting direct and indirect parametrization, ID generation, and scope resolution.

• Supports validation for argument names, indirectness, and IDs.

• Manages internal list of CallSpec2 instances representing all parameter combinations.

• Integrates with fixture system by creating pseudo-fixturedefs for direct parameters.

Important Implementation Details and Algorithms

• Collection Filtering: Uses glob patterns and prefixes from configuration (python_files, python_classes, python_functions) to filter files, classes, and functions that qualify as tests.

• Parametrization: The Metafunc.parametrize() method handles multiple calls to parametrize, producing the Cartesian product of parameter sets.

• Setup/Teardown Integration: Legacy xunit-style setup/teardown functions (setup_module, setup_class, setup_method, teardown_function, etc.) are registered as autouse fixtures, ensuring compatibility with pytest’s fixture system.

• Unique Test IDs Generation: The IdMaker class ensures each parameter set has a unique ID for naming test instances, avoiding collisions using suffixes.

• Async Test Function Handling: Detects async test functions and raises informative failures directing users to appropriate plugins.

• Safe Module Imports: The importtestmodule() function attempts to import test modules carefully, catching import errors, syntax errors, and import path mismatches, and presenting clear errors.

• Node Tree Construction: Uses from_parent() factory methods and mixin classes (PyobjMixin) to build a hierarchical tree of Package, Module, Class, and Function nodes.

Interactions with Other Parts of the System

• _pytest.nodes: Base classes for all nodes (Node, Collector, Item) are extended here to create Python-specific collectors and items.

• _pytest.fixtures: The fixture system is integrated deeply; fixtures are parsed, and parametrized arguments may be passed to fixture functions.

• _pytest.config: Accesses pytest configuration options and ini values to determine test discovery patterns and behaviors.

• _pytest.mark: Manages test markers, including parametrization marks.

• _pytest.outcomes: Used to raise test failures and skips during collection and execution phases.

• Hooks: Implements hooks like pytest_pyfunc_call, pytest_collect_directory, pytest_collect_file, and pytest_pycollect_makeitem to allow plugin extension.

• Session: Each collector is associated with a test session that manages the overall test lifecycle.

Usage Examples

Parametrizing a Test Function

@pytest.mark.parametrize("input,expected", [(1, 2), (3, 4)])
def test_increment(input, expected):
    assert increment(input) == expected

During collection, the `Metafunc.parametrize()` method is called to generate two calls of `test_increment` with different arguments.

Implementing a Custom Test Collector Plugin

def pytest_collect_file(file_path, parent):
    if file_path.suffix == ".py" and file_path.name.startswith("custom_"):
        return Module.from_parent(parent, path=file_path)

Mermaid Diagram: Python Test Collection Class Structure

classDiagram
    class PyobjMixin {
        +module
        +cls
        +instance
        +obj
        +getmodpath(stopatmodule: bool, includemodule: bool)
        +reportinfo()
    }
    class PyCollector {
        +funcnamefilter(name: str)
        +isnosetest(obj: object)
        +classnamefilter(name: str)
        +istestfunction(obj: object, name: str)
        +istestclass(obj: object, name: str)
        +collect()
        +_genfunctions(name: str, funcobj)
    }
    class Module {
        +_getobj()
        +collect()
        +_register_setup_module_fixture()
        +_register_setup_function_fixture()
    }
    class Package {
        +setup()
        +collect()
    }
    class Class {
        +newinstance()
        +collect()
        +_register_setup_class_fixture()
        +_register_setup_method_fixture()
    }
    class Function {
        +runtest()
        +setup()
        +repr_failure(excinfo)
    }
    PyobjMixin <|-- PyCollector
    PyCollector <|-- Module
    PyCollector <|-- Class
    nodes.Directory <|-- Package
    PyobjMixin <|-- Function

Summary

The [python.py](/projects/286/67218) file is a fundamental pytest module that implements the discovery and collection of Python tests from files, packages, classes, and functions. It uses naming conventions, configuration options, and introspection to identify test units, supports parametrization to generate multiple test calls, and bridges legacy xunit-style setup/teardown into the fixture system. This module forms the backbone of pytest's Python test collection and prepares test items for execution within the framework.

End of Documentation for python.py