pytester.py

Overview

`pytester.py` provides a rich set of tools, fixtures, and helper classes designed to facilitate testing of **pytest itself** and **pytest plugins**. It offers an isolated environment for running pytest test runs—either in-process or subprocess—enabling black-box style testing of pytest features, plugins, and configurations. This module provides mechanisms to create test files and directories, run pytest with various options, capture and analyze output, and introspect hook calls and test reports.

Key functionalities include:

• Creating temporary test files and directories (Python modules, config files, etc.).

• Running pytest runs inline (in-process) or as subprocesses.

• Capturing and analyzing pytest hook calls via HookRecorder.

• Utilities for matching and asserting text output (LineMatcher, LineComp).

• Facilities to snapshot and restore interpreter state (sys.modules, sys.path).

• A fixture pytester exposing the Pytester class for test writers.

• Support for checking file descriptor leaks on platforms with lsof.

This file is a core utility in the pytest testing ecosystem, enabling pytest's own tests and plugin authors to verify pytest behavior reliably.

Detailed Documentation

Module-level Functions

pytest_addoption(parser: Parser) -> None

Adds command-line options and ini settings related to the pytester utilities:

• --lsof: Enable file descriptor leak checks if lsof is available.

• --runpytest: Choose whether inner pytest runs execute "inprocess" or via "subprocess".

• pytester_example_dir: ini option to specify example files directory.

pytest_configure(config: Config) -> None

Configures the pytester plugin, registering [LsofFdLeakChecker](/projects/286/67442) if needed, and adds a marker [pytester_example_path](/projects/286/67470) to help locate test example files.

Class: LsofFdLeakChecker

A platform-specific file descriptor leak checker using `lsof`. It runs before and after each test to detect leaked file descriptors.

• Methods:

  • get_open_files() -> list[tuple[str, str]]: Uses lsof to get a list of open file descriptors and their paths, filtering out irrelevant ones.

  • matching_platform() -> bool: Checks if the platform supports running lsof.

  • pytest_runtest_protocol(item: Item) -> Generator: Hook wrapper to detect FD leaks around test runs, issuing warnings if leaks are detected.

Fixture: _pytest

Returns a [PytestArg](/projects/286/67266) helper, which exposes a [gethookrecorder()](/projects/286/67271) method to capture hook calls during pytest runs.

Class: PytestArg

Helper class returned by `_pytest` fixture.

• __init__(request: FixtureRequest)

• gethookrecorder(hook) -> HookRecorder: Returns a HookRecorder for the given hook object.

Utility Function: get_public_names(values: Iterable[str]) -> list[str]

Filters an iterable of names and returns only those that do not start with an underscore (public names).

Class: RecordedHookCall

Represents a single recorded hook call, storing the name of the hook and its keyword arguments as attributes.

• __init__(name: str, kwargs)

• Attributes: dynamically created from kwargs.

• __repr__(): Displays the hook name and arguments.

Usage example:

calls = hook_recorder.getcalls("pytest_runtest_setup")
assert calls[0].item is an_item

Class: HookRecorder

Records all hook calls made by a `PytestPluginManager` and provides utilities to query and assert them.

• __init__(pluginmanager: PytestPluginManager, *, _ispytest=False)

• finish_recording(): Stops recording hooks.

• getcalls(names: str | Iterable[str]) -> list[RecordedHookCall]: Return recorded calls matching one or more hook names.

• assert_contains(entries: Sequence[tuple[str, str]]): Assert certain hook calls occurred with conditions.

• popcall(name: str) -> RecordedHookCall: Remove and return first call matching name.

• getcall(name: str) -> RecordedHookCall: Return exactly one call matching name (asserts uniqueness).

• getreports(names=...) -> Sequence[CollectReport | TestReport]: Extract test reports from recorded hook calls.

• matchreport(...) -> CollectReport | TestReport: Return a specific test report matching criteria.

• getfailures(names=...) -> Sequence[CollectReport | TestReport]: Get failed test reports.

• getfailedcollections() -> Sequence[CollectReport]

• listoutcomes() -> tuple: Returns (passed, skipped, failed) reports.

• countoutcomes() -> list[int]: Count of test outcomes.

• assertoutcome(passed=0, skipped=0, failed=0): Assert expected test outcomes.

• clear(): Clear recorded calls.

Fixtures: linecomp, LineMatcher

• linecomp: Returns a LineComp instance for asserting that a stream linearly contains a sequence of strings.

• LineMatcher: Provides access to LineMatcher class, useful for matching and asserting lines of text output.

Class: RunResult

Encapsulates the result of running a pytest command or subprocess.

• __init__(ret, outlines, errlines, duration)

• Attributes:

  • ret: Return code (ExitCode or int).

  • outlines: List of stdout lines.

  • errlines: List of stderr lines.

  • stdout: LineMatcher instance for stdout.

  • stderr: LineMatcher instance for stderr.

  • duration: Duration in seconds.

• parseoutcomes() -> dict[str,int]: Parse pytest terminal summary from stdout lines.

• assert_outcomes(...): Assert expected test outcomes were reported.

Classes: SysModulesSnapshot, SysPathsSnapshot

Snapshot and restore the state of `sys.modules` and `sys.path` respectively to isolate test runs and prevent pollution between tests.

Class: Pytester

The central class of this module, providing comprehensive facilities for pytest plugin and pytest core testing.

**Key features:**

• Creates temporary filesystem layouts for tests.

• Writes source files, config files (conftest.py, tox.ini, pyproject.toml).

• Runs pytest in-process or subprocess with configurable plugins.

• Provides access to pytest collection nodes and test items.

• Records hook calls and test reports.

• Runs arbitrary subprocess commands and captures output.

• Supports spawning processes via pexpect.

• Manages interpreter state snapshots and environment isolation.

• Provides helper methods to create python packages and directories.

**Initialization:**

Pytester(
    request: FixtureRequest,
    tmp_path_factory: TempPathFactory,
    monkeypatch: MonkeyPatch,
    *,
    _ispytest: bool = False,
)

**Important properties and methods:**

• path: Temporary test directory path.

• makefile(ext, *lines, **kwargs): Create one or more text files with given extension.

• makepyfile(*args, **kwargs): Create Python source files.

• makeconftest(source): Create a conftest.py file.

• makeini(source): Create a .ini config file.

• getinicfg(source): Parse and return the pytest section from ini config.

• mkpydir(name): Create a Python package directory.

• copy_example(name): Copy example files from configured example directory.

• getnode(config, arg): Return pytest collection node for a source file.

• getpathnode(path): Like getnode but configures pytest automatically.

• getitem(source, funcname): Get a test item for a function in a source module.

• getitems(source): Get all test items from a source module.

• getmodulecol(source, configargs=(), withinit=False): Get module collector node.

• collect_by_name(modcol, name): Get collection node by name.

• runpytest(*args, **kwargs): Run pytest (in-process or subprocess).

• inline_run(*args, plugins=(), no_reraise_ctrlc=False): Run pytest inline.

• runpytest_inprocess(*args, **kwargs): Run pytest in-process and return RunResult.

• runpytest_subprocess(*args, timeout=None): Run pytest in a subprocess.

• runpython(script): Run a python script.

• runpython_c(command): Run a python command via -c.

• spawn(cmd, expect_timeout=10.0): Run a command with pexpect.

• spawn_pytest(string, expect_timeout=10.0): Run pytest with pexpect.

Class: LineComp

Helper class wrapping a `StringIO` used to assert that the stream contains lines in a linear sequence.

• Method: assert_contains_lines(lines2)

Class: LineMatcher

Utility class for flexible line-by-line matching of text output using glob (`fnmatch`) or regex.

• __init__(lines: list[str])

• __str__(): Return original text.

• fnmatch_lines(lines2, *, consecutive=False): Assert lines matching glob patterns in output.

• re_match_lines(lines2, *, consecutive=False): Assert lines matching regex patterns.

• fnmatch_lines_random(lines2): Assert lines present in any order with glob.

• re_match_lines_random(lines2): Assert lines present in any order with regex.

• no_fnmatch_line(pat): Assert no lines matching glob pattern.

• no_re_match_line(pat): Assert no lines matching regex pattern.

• get_lines_after(fnline): Return lines after a matching line.

• Internal logging and failure helper methods.

Implementation Details and Algorithms

• File Descriptor Leak Detection: Uses lsof output parsing to detect which file descriptors are opened before and after each test, warning about leaks.

• Hook Recording: HookRecorder uses Pytest's hook call monitoring API to record all hooks called during a pytest run.

• Isolation: Uses snapshots of sys.modules and sys.path to restore interpreter state after inline pytest runs, preventing state leakage.

• File Creation: Utility methods convert given strings or bytes into files with proper encoding and structure, supporting multiple files per call.

• Running pytest: Supports running pytest both inline (via pytest.main) or subprocess, controlled by command line option.

• Text Matching: LineMatcher uses both glob and regex matching, supports consecutive or random order, and produces detailed error logs on failure.

• pexpect Support: Allows spawning pytest or commands under pexpect for interactive testing scenarios.

Interaction with Other Parts of the System

• Imports many core pytest modules and internal APIs (_pytest.config, _pytest.main, _pytest.nodes, etc.) to access pytest internals.

• Uses iniconfig for ini file parsing.

• Exposes a fixture pytester to pytest test functions, enabling plugin and pytest core tests to use this functionality.

• Integrates with the pytest plugin manager and hook system to provide hook recording and test report introspection.

• Supports interaction with pytest-xdist and other plugins via shared fixtures and hook recording.

• Supports external tools like lsof for FD leak detection and pexpect for process interaction.

• Uses temporary directories and environment variable patching to isolate test runs.

Visual Diagram: Class Diagram

classDiagram
    class Pytester {
        +path: Path
        +makefile(ext, *args, **kwargs) Path
        +makepyfile(*args, **kwargs) Path
        +makeconftest(source) Path
        +makeini(source) Path
        +getnode(config, arg) Collector|Item
        +getitem(source, funcname) Item
        +runpytest(*args, **kwargs) RunResult
        +inline_run(*args, plugins=(), no_reraise_ctrlc=False) HookRecorder
        +runpython(script) RunResult
        +spawn(cmd, expect_timeout=10.0) pexpect.spawn
        +mkdir(name) Path
        +mkpydir(name) Path
        +copy_example(name) Path
    }

    class HookRecorder {
        +calls: list[RecordedHookCall]
        +getcalls(names) list[RecordedHookCall]
        +assert_contains(entries)
        +popcall(name) RecordedHookCall
        +getcall(name) RecordedHookCall
        +getreports(names) list
        +getfailures(names) list
        +assertoutcome(passed, skipped, failed)
    }

    class RecordedHookCall {
        +_name: str
        +__repr__()
    }

    class RunResult {
        +ret: ExitCode|int
        +outlines: list[str]
        +errlines: list[str]
        +stdout: LineMatcher
        +stderr: LineMatcher
        +duration: float
        +parseoutcomes() dict
        +assert_outcomes(...)
    }

    class LineMatcher {
        +lines: list[str]
        +fnmatch_lines(lines2, consecutive=False)
        +re_match_lines(lines2, consecutive=False)
        +no_fnmatch_line(pat)
        +no_re_match_line(pat)
    }

    Pytester --> HookRecorder : creates
    HookRecorder --> RecordedHookCall : records
    Pytester --> RunResult : returns results
    RunResult --> LineMatcher : stdout, stderr

Usage Examples

Example: Creating a Python file and running pytest inline

def test_example(pytester):
    pytester.makepyfile("""
        def test_always_passes():
            assert True
    """)
    result = pytester.runpytest()
    result.stdout.fnmatch_lines(["*1 passed*"])

Example: Using HookRecorder to assert hook calls

def test_hook_calls(_pytest):
    hookrec = _pytest.gethookrecorder(pytester_hook)
    # run some pytest actions
    calls = hookrec.getcalls("pytest_runtest_setup")
    assert calls
    assert hasattr(calls[0], "item")

Example: Using LineMatcher for output assertions

def test_output_matching():
    output_lines = [
        "Starting tests",
        "test_foo.py::test_bar PASSED",
        "=== 1 passed in 0.01s ==="
    ]
    matcher = LineMatcher(output_lines)
    matcher.fnmatch_lines([
        "test_foo.py::* PASSED",
        "*1 passed*"
    ])

Summary

`pytester.py` is a comprehensive testing utility module designed to facilitate testing pytest itself and pytest plugins by providing an isolated environment, file management, process running capabilities, detailed hook recording, and output assertion helpers. Its core class `Pytester` along with `HookRecorder`, `RunResult`, and `LineMatcher` provide a powerful toolkit for black-box and white-box testing of pytest behavior. This module integrates deeply with pytest internals and external tools to achieve reliable and reproducible test environments.

End of Documentation for pytester.py