test_pytester.py

Overview

The [test_pytester.py](/projects/286/67377) file is a comprehensive test suite designed to verify the functionality, robustness, and integration of the `pytester` testing utility within the `pytest` framework. The file primarily contains unit and integration tests that:

• Validate core features of the pytester API, such as creating hook recorders, running tests in subprocesses, and managing temporary test files.

• Ensure proper handling of edge cases including unicode input, timeout behavior, and subprocess communication.

• Test internal mechanisms for snapshotting and restoring Python module and path states (SysModulesSnapshot and SysPathsSnapshot).

• Confirm correct parsing of pytest output summaries and outcomes, including support for warnings, deselected tests, and expected failures (xfail).

• Check the behavior of auxiliary utilities like LineMatcher for matching lines in output.

• Validate environment isolation features such as temporary environment variables and home directory overrides during test runs.

This file interacts extensively with the `_pytest.pytester` module and other pytest internals, serving as a critical validation layer for `pytester` functionality to ensure reliable test execution and plugin support.

Detailed Explanations

Key Classes and Functions

**Note:** The file mostly contains test functions and test classes; the main entities tested come from the `_pytest.pytester` module such as `Pytester`, `HookRecorder`, `SysModulesSnapshot`, and `SysPathsSnapshot`.

test_make_hook_recorder(pytester: Pytester) -> None

• Purpose: Tests creation and basic usage of a HookRecorder to track pytest hook calls and failures.

• Parameters:

  • pytester: Fixture providing Pytester instance.

• Behavior:

  • Creates a hook recorder for a test item.

  • Simulates test reports including passed, failed, and skipped.

  • Checks that failures are correctly recorded and counted.

  • Ensures unregistering and clearing recorder functionality works.

• Usage Example:

  def test_make_hook_recorder(pytester):
      item = pytester.getitem("def test_func(): pass")
      recorder = pytester.make_hook_recorder(item.config.pluginmanager)
      # ... further assertions as above
  

• Implementation Details:

  • The test fakes pytest_runtest_logreport hook calls with report objects to simulate test outcomes.

  • Verifies the hook recorder accumulates and reports failures correctly.

test_parseconfig(pytester: Pytester) -> None

• Purpose: Verifies that multiple calls to pytester.parseconfig() return distinct config instances.

• Usage: Ensures isolation between pytest config objects.

test_pytester_runs_with_plugin(pytester: Pytester) -> None

• Purpose: Validates that pytester can run tests that declare pytest_plugins and that the plugin is recognized.

• Details: Creates a test file with pytest_plugins = "pytester" and runs pytest, checking for a passing test.

test_pytester_with_doctest(pytester: Pytester) -> None

• Purpose: Confirms that pytester works correctly when used inside doctests, where request.function can be None.

• Details: Creates a doctest file and runs pytest with --doctest-modules.

test_runresult_assertion_on_xfail and test_runresult_assertion_on_xpassed

• Purpose: Check pytester correctly handles the xfail and xpassed test outcomes.

• Details: Create tests marked with @pytest.mark.xfail that either fail or pass, asserting the resulting outcome codes.

test_xpassed_with_strict_is_considered_a_failure

• Purpose: Verifies that an xfail(strict=True) test that unexpectedly passes is treated as a failure.

make_holder() -> tuple[type, ModuleType]

• Purpose: Utility to create a test hook spec holder class and module with example hook methods.

• Returns: A tuple of a class and a module defining pytest_xyz hooks.

test_hookrecorder_basic(holder)

• Purpose: Tests that HookRecorder records calls to hooks defined in the holder.

• Parameters: holder is parameterized over the class and module from make_holder.

• Details: Adds hooks to a PytestPluginManager, triggers hooks, and verifies recorder captures them.

test_makepyfile_unicode and test_makepyfile_utf8

• Purpose: Verify that pytester.makepyfile accepts unicode strings and UTF-8 encoded bytes correctly.

TestInlineRunModulesCleanup

• Purpose: Tests related to the cleanup of modules after inline test runs.

• Key Tests:

  • Ensures test modules are not re-imported incorrectly.

  • Spies on SysModulesSnapshot to verify correct restoring of sys.modules.

  • Checks preservation of certain modules during restore.

  • Confirms that external modules imported in tests are not cleaned up unexpectedly.

TestSysModulesSnapshot

• Purpose: Unit tests for the SysModulesSnapshot class.

• Functionality Tested:

  • Removing modules added after snapshot.

  • Adding back modules removed after snapshot.

  • Reloading modules replaced after snapshot.

  • Custom preservation predicate for modules.

  • Ensures snapshot restores sys.modules correctly even if container replaced.

TestSysPathsSnapshot

• Purpose: Unit tests for SysPathsSnapshot which snapshots and restores sys.path and sys.meta_path.

• Tests:

  • Various mutations of sys.path and restoring it.

  • Preservation of path containers after restore.

test_pytester_subprocess and test_pytester_subprocess_via_runpytest_arg

• Purpose: Confirm pytester can run tests in subprocesses and via runpytest argument.

test_unicode_args

• Purpose: Verify pytester handles unicode command line arguments.

test_pytester_run_no_timeout, test_pytester_run_with_timeout, and test_pytester_run_timeout_expires

• Purpose: Test subprocess execution respects timeout parameters, raises on timeout.

test_linematcher_* functions

• Purpose: Test LineMatcher utility for matching output lines using fnmatch or regex:

  • Correct exceptions are raised on mismatches.

  • Handles non-list input types gracefully.

  • Supports consecutive matching mode.

  • Supports checking for no matches of certain patterns.

  • Provides string representation output.

test_pytest_addopts_before_pytester

• Purpose: Ensures environment variable PYTEST_ADDOPTS is cleared before pytester fixture is used to avoid contamination.

test_run_stdin, test_popen_stdin_pipe, test_popen_stdin_bytes, test_popen_default_stdin_stderr_and_stdin_None

• Purpose: Test pytester.run and pytester.popen subprocess helpers with various stdin configurations, including pipes and byte input.

test_spawn_uses_tmphome

• Purpose: Verifies that spawned pytest subprocesses use pytester's temporary home directory environment (HOME), along with custom environment variables.

test_run_result_repr

• Purpose: Checks the string representation of RunResult instances for known and unknown exit codes.

test_pytester_outcomes_with_multiple_errors

• Purpose: Validates that multiple errors from fixtures are properly reported by pytester.

test_parse_summary_line_always_plural

• Purpose: Ensures summary line parser always returns plural noun keys for test outcome counts.

test_makefile_joins_absolute_path and test_pytester_makefile_dot_prefixes_extension_with_warning

• Purpose: Test pytester.makefile behavior with absolute paths and file extensions, ensuring correct warnings/errors.

test_pytester_assert_outcomes_warnings and test_pytester_outcomes_deselected

• Purpose: Test that the assertion of outcomes in pytester handles warnings and deselected tests correctly.

test_pytester_subprocess_with_string_plugins and test_pytester_subprocess_with_non_string_plugins

• Purpose: Verify that pytester.runpytest_subprocess accepts string plugin names but raises an error on plugin instances (non-strings).

Important Implementation Details and Algorithms

• HookRecorder:

  • Captures hook calls for inspection.

  • Tracks test outcomes (passed, failed, skipped).

  • Provides both list and count summaries of outcomes and failed collections.

  • Can be unregistered and cleared for state reset.

• SysModulesSnapshot & SysPathsSnapshot:

  • Snapshot current sys.modules and sys.path / sys.meta_path states.

  • On restore, revert any additions, removals, or modifications.

  • Support preservation predicates or containers to avoid restoring certain modules or path entries.

  • Used to isolate test runs and prevent pollution of the global interpreter state.

• LineMatcher:

  • Matches lines of output using fnmatch or regex patterns.

  • Provides detailed failure messages with unmatched lines.

  • Supports consecutive matching mode.

  • Robust against incorrect input types.

• Timeout Handling:

  • Uses _pytest.timing.Instant to measure elapsed time.

  • Raises TimeoutExpired if test subprocess exceeds allowed time.

• Environment Isolation:

  • Uses monkeypatching to set environment variables like HOME.

  • Ensures that PYTEST_ADDOPTS is not inherited in subprocess runs.

Interaction with Other Parts of the System

• The file heavily tests the pytester fixture and related classes from the _pytest.pytester module.

• Uses PytestPluginManager and hooks from _pytest.config to simulate and test plugin hook interactions.

• Interacts with _pytest.timing for timeout measurement.

• Utilizes pytest core testing mechanisms for assertions, fixture injection, and error handling.

• The tests help ensure that the pytester utility integrates seamlessly into the pytest testing lifecycle and supports subprocess-based test runs, plugin management, and output parsing.

Visual Diagram

classDiagram
    class Pytester {
        +makepyfile(content)
        +runpytest(args)
        +runpytest_subprocess(args)
        +inline_run(args)
        +make_hook_recorder(pluginmanager)
        +popen(args)
        +spawn_pytest(path)
        +syspathinsert()
    }

    class HookRecorder {
        +getfailures()
        +listoutcomes()
        +countoutcomes()
        +getfailedcollections()
        +popcall(name)
        +unregister()
        +clear()
    }

    class SysModulesSnapshot {
        -__snapshot: dict
        +restore()
    }

    class SysPathsSnapshot {
        +restore()
    }

    class LineMatcher {
        +fnmatch_lines(lines, consecutive=False)
        +re_match_lines(lines, consecutive=False)
        +no_fnmatch_line(pattern)
        +no_re_match_line(pattern)
        +_getlines(input)
        +__str__()
    }

    Pytester --> HookRecorder : creates
    Pytester --> SysModulesSnapshot : uses
    Pytester --> SysPathsSnapshot : uses
    Pytester --> LineMatcher : uses for output matching
    HookRecorder --> PytestPluginManager : hooks into

Summary

The [test_pytester.py](/projects/286/67377) file is a thorough test suite validating the `pytester` testing utility's features, including hook recording, subprocess test execution, module and path snapshotting, output matching, and environment isolation. It ensures that `pytester` behaves correctly under various scenarios and integrates properly with pytest’s plugin and test-running infrastructure. The file’s tests cover both normal and edge cases, making it a critical component for maintaining the reliability and functionality of pytest's testing utilities.