doctest.py

Overview

The [doctest.py](/projects/286/67506) file is a core component of the pytest framework that provides comprehensive support for discovering, collecting, running, and reporting **doctests** embedded within Python modules (`.py` files) and standalone text files (e.g., `.txt`, `.rst`). Doctests are interactive Python examples found in documentation strings or text files that serve as both documentation and executable tests.

This module enables pytest to:

• Discover doctests automatically in source code and text files based on user configuration.

• Execute doctests with enhanced control, including continuation on failure.

• Integrate pytest fixtures into doctest namespaces, allowing powerful test setups.

• Provide rich and configurable failure reporting with multiple diff styles.

• Handle special platform-specific behaviors (e.g., macOS output capturing).

• Extend the standard doctest output checker to tolerate differences in unicode/byte prefixes and floating-point precision.

By integrating doctests seamlessly into the pytest ecosystem, this file allows users to run doctests alongside pytest tests, benefiting from pytest’s fixtures, reporting, and test selection mechanisms.

Main Classes and Functions

1. pytest_addoption(parser: Parser) -> None

**Purpose:** Registers command-line options and ini-file configuration entries related to doctest collection and execution.

**Key Options Registered:**

• --doctest-modules: Run doctests in all .py modules.

• --doctest-report: Choose output diff format on failure (none, cdiff, ndiff, udiff, only_first_failure).

• --doctest-glob: Glob patterns for doctest text files.

• --doctest-ignore-import-errors: Ignore import errors during doctest collection.

• --doctest-continue-on-failure: Continue running all examples in a doctest after the first failure.

**Usage:** This function is automatically called by pytest during plugin registration.

2. pytest_collect_file(file_path: Path, parent: Collector) -> DoctestModule | DoctestTextfile | None

**Purpose:** Hook to determine if a file should be collected as a doctest. Returns a collector instance if the file contains doctests, or [None](/projects/286/67505) otherwise.

**Logic:**

• If the file is a .py module and --doctest-modules is enabled, and the file is not setup.py or __main__.py, collect as a DoctestModule.

• If the file matches doctest glob patterns (e.g., test*.txt), collect as a DoctestTextfile.

• Otherwise, skip collection.

3. class DoctestItem(Item)

Represents a single doctest within a module or text file.

**Constructor Parameters:**

• name (str): Name identifying the doctest.

• parent (DoctestTextfile | DoctestModule): Parent collector.

• runner (doctest.DocTestRunner): Runner instance to execute the doctest.

• dtest (doctest.DocTest): The doctest object to run.

**Key Methods:**

• setup(): Prepares the test environment.

  • Fills pytest fixtures.

  • Injects the doctest_namespace fixture and other fixtures into the doctest’s globals.

• runtest(): Executes the doctest.

  • Checks if all examples are skipped.

  • Disables output capturing on macOS to ensure stdout is visible.

  • Runs the doctest, collecting failures.

  • Raises MultipleDoctestFailures if any failures occur.

• repr_failure(excinfo): Customizes failure reporting.

  • Aggregates multiple doctest failures.

  • Formats output diffs according to user-chosen style.

  • Shows relevant source lines with line numbers.

• reportinfo(): Returns information for reporting (file path, line number, and test name).

**Example Usage:**

# Typically, created by the collection mechanism:
doctest_item = DoctestItem.from_parent(
    parent=some_doctest_module,
    name="module_name.test_function",
    runner=runner_instance,
    dtest=doctest_obj
)

doctest_item.setup()
doctest_item.runtest()

4. class DoctestTextfile(Module)

Collector for doctests found in external text files (like `.txt` or `.rst`).

**Key Method:**

• collect() -> Iterable[DoctestItem]:

  • Reads the file content with configured encoding.

  • Parses the doctests using doctest.DocTestParser.

  • Yields DoctestItem instances for each found doctest.

5. class DoctestModule(Module)

Collector for doctests inside Python `.py` modules.

**Key Method:**

• collect() -> Iterable[DoctestItem]:

  • Attempts to import the module.

  • Uses a customized DocTestFinder subclass (MockAwareDocTestFinder) to find doctests in the module.

  • Yields DoctestItem instances for each doctest found.

**Implementation Details:**

• MockAwareDocTestFinder works around Python stdlib issues with line number detection and mock object handling.

• Supports autouse fixtures by parsing factories from the session’s fixture manager.

6. class MultipleDoctestFailures(Exception)

Exception used to aggregate multiple doctest failures within a single doctest run.

• failures: List of doctest.DocTestFailure objects.

7. _init_runner_class() -> type[doctest.DocTestRunner]

Returns a customized doctest runner class `PytestDoctestRunner`:

• Inherits doctest.DebugRunner.

• Overrides:

  • report_failure: Collects failures instead of stopping immediately (based on continue_on_failure).

  • report_unexpected_exception: Handles exceptions and optionally continues or raises immediately.

• Supports continue_on_failure flag.

• Prevents debugger quitting exceptions from being swallowed.

8. _get_runner(...) -> doctest.DocTestRunner

Returns a singleton instance of the doctest runner (`PytestDoctestRunner`), ensuring lazy import of `doctest`.

9. _init_checker_class() -> type[doctest.OutputChecker]

Returns a customized output checker class `LiteralsOutputChecker` with enhanced output comparison abilities:

• Ignores Unicode prefix (u'') if configured.

• Ignores Byte prefix (b'') if configured.

• Compares floating-point numbers approximately, tolerating minor precision differences.

10. _get_checker() -> doctest.OutputChecker

Returns a singleton instance of the customized `LiteralsOutputChecker`.

11. doctest_namespace() -> dict[str, Any] (pytest fixture)

Provides a mutable dictionary injected into the namespace of all doctests.

**Usage Example:**

@pytest.fixture(autouse=True)
def add_np(doctest_namespace):
    doctest_namespace["np"] = numpy

This enables doctests to use fixtures and shared objects.

12. Helper Functions

• _is_setup_py(path: Path) -> bool: Detects if a file is setup.py with setuptools/distutils content.

• _is_main_py(path: Path) -> bool: Detects if a file is __main__.py.

• _is_doctest(config, path, parent) -> bool: Checks if a file matches doctest glob patterns.

• _check_all_skipped(test: doctest.DocTest): Skips a test if all examples have the +SKIP option.

• _is_mocked(obj: object) -> bool: Determines if an object is a mock to prevent recursion in unwrapping.

• _patch_unwrap_mock_aware(): Context manager to patch inspect.unwrap to be mock-aware.

• _get_report_choice(key: str) -> int: Maps user-friendly report choice strings to doctest constants.

• _get_flag_lookup() -> dict[str, int]: Maps option flag names to doctest constants.

• get_optionflags(config: Config) -> int: Combines option flags from pytest config into bitmask.

• _get_continue_on_failure(config: Config) -> bool: Determines whether to continue after first failure.

Implementation Details and Algorithms

Doctest Collection

• Uses pytest’s collection hooks to identify .py modules and text files for doctest discovery.

• Skips special files like setup.py and __main__.py to avoid false positives.

• Uses doctest.DocTestFinder for module doctests and doctest.DocTestParser for text files.

• Applies fixups and workarounds for Python bugs and mock objects through a subclassed finder.

Doctest Execution

• Uses a custom runner PytestDoctestRunner extending doctest.DebugRunner that collects failures in a list rather than stopping immediately.

• Supports the continue_on_failure setting to either collect all failures or abort on the first.

• Injects fixtures via the doctest_namespace fixture into the doctest globals before running.

• On macOS, disables pytest’s output capturing to prevent lost stdout from doctests.

• Aggregates multiple failures and raises a MultipleDoctestFailures exception.

Failure Reporting

• Overrides repr_failure to provide detailed diffs using the selected diff style (e.g., udiff).

• Shows source lines with line numbers for context around the failed example.

• Supports multiple failure aggregation with clear labeling.

• Uses regexes and approximate comparison in the LiteralsOutputChecker to accommodate common doctest output variations.

Interaction with Other Parts of the System

• pytest Collection System: Hooks into file collection, returning DoctestModule or DoctestTextfile collectors.

• pytest Fixture System: Uses fixtures to inject shared namespaces into doctests (doctest_namespace fixture).

• pytest Configuration: Reads ini options and command-line flags for doctest behavior.

• pytest Reporting: Integrates with pytest’s terminal reporting to present doctest failures elegantly.

• pytest Outcomes: Uses pytest’s skip and OutcomeException for flow control.

• Platform Compatibility: Handles macOS-specific output capturing issues.

• Lazy Imports: Imports doctest only when needed to minimize startup overhead.

Visual Diagram

flowchart TD
    A[pytest_collect_file]
    A -->|.py & --doctest-modules| B[DoctestModule Collector]
    A -->|matches glob| C[DoctestTextfile Collector]
    B --> D[MockAwareDocTestFinder]
    C --> E[doctest.DocTestParser]
    D --> F[DoctestItem]
    E --> F
    F --> G[DoctestItem.setup()]
    G --> H[Inject fixtures into doctest.globs]
    H --> I[DoctestItem.runtest()]
    I --> J[PytestDoctestRunner.run()]
    J -->|failures| K[MultipleDoctestFailures Exception]
    K --> L[DoctestItem.repr_failure()]
    L --> M[Formatted failure report with diff]

Summary

The [doctest.py](/projects/286/67506) file is the cornerstone of pytest’s support for running doctests. It provides:

• Transparent collection of doctests from Python modules and text files.

• Custom doctest runner and output checker with enhanced capabilities.

• Integration with pytest fixtures for richer test environments.

• Robust failure aggregation and user-friendly reporting.

• Configurable doctest behavior via command-line options and ini settings.

• Platform-specific handling and extensive compatibility workarounds.

This integration empowers users to maintain and test documentation examples confidently within the pytest ecosystem, ensuring high-quality, tested documentation as part of their development workflow.