Doctest Support

Purpose

Doctest Support addresses the need to seamlessly collect and run doctests embedded in Python modules and standalone text files within the pytest framework. Doctests are interactive Python examples embedded in docstrings or text files that serve both as documentation and as simple tests. This subtopic enables pytest users to leverage doctests alongside other test styles, ensuring consistent test discovery, execution, and reporting without requiring separate tooling.

Functionality

The core functionality revolves around discovering doctests in `.py` modules and `.txt` or `.rst` files (matching configurable patterns), executing these doctests with fine-grained control, and integrating their results into pytest's test outcome reporting.

Key Workflows

• Doctest Collection:

  • For Python files, when the --doctest-modules option is enabled, the system collects doctests from modules, excluding special files like setup.py or __main__.py.

  • For text files, files matching patterns such as test*.txt are collected as doctest text files.

  • Collection returns DoctestModule or DoctestTextfile objects that collect individual DoctestItems representing each discovered doctest.

• Doctest Execution:

  • Each DoctestItem runs the contained doctest using a custom runner subclassed from Python's doctest.DebugRunner.

  • The runner supports options like continuing on failure (running all examples even if some fail) and detailed failure reporting.

  • The test globals are enriched with fixture-provided namespaces, allowing doctests to access pytest fixtures indirectly.

• Failure Reporting:

  • Failures produce detailed, user-friendly diffs with configurable diff styles (udiff, cdiff, ndiff, or no diff).

  • Multiple failures per doctest are aggregated and reported together, providing comprehensive feedback.

• Output Handling:

  • Special handling for macOS disables pytest’s output capturing during doctest runs to prevent loss of stdout, ensuring doctest output is visible.

Configuration and Option Flags

• Users can control doctest behavior via command-line options and ini settings, such as:

  • --doctest-modules to enable or disable module doctests.

  • --doctest-glob to specify file patterns for text file doctests.

  • --doctest-report to select the diff style for failures.

  • doctest_optionflags ini entry to specify doctest option flags like ELLIPSIS or NORMALIZE_WHITESPACE.

  • --doctest-continue-on-failure to decide whether to stop after the first failure or continue testing.

Enhanced Output Checking

• A custom OutputChecker subclass extends standard doctest output comparison with options to ignore Unicode (u'') and byte (b'') string prefixes and to tolerate minor floating-point precision differences, increasing robustness across Python versions and environments.

Relationship to Parent Topic and Other Subtopics

Doctest Support complements the broader **Integration with Other Test Frameworks** topic by enabling pytest to run doctests natively, alongside unittest-based tests and standard pytest tests. This integration ensures:

• Unified test collection and execution flow using pytest’s collection and item model, enabling doctests to participate in fixture management and reporting mechanisms.

• Consistent failure representation with pytest’s rich terminal output and traceback formatting.

• Shared configuration and plugin infrastructure, allowing doctest behavior to be extended or customized using pytest's plugin system.

• Synergy with the Fixture Management subtopic, as doctest globals can be supplemented with fixtures via the doctest_namespace fixture, bridging doctest and pytest fixture ecosystems.

This subtopic introduces advanced doctest runner customization and output checking that are not covered in the parent topic or other subtopics, focusing specifically on the unique challenges of doctest execution, error aggregation, and output comparison.

Code Snippet Illustrations

Customized Doctest Runner Class

A specialized runner aggregates failures and supports continuing execution after failures:

class PytestDoctestRunner(doctest.DebugRunner):
    def __init__(self, ..., continue_on_failure=True):
        super().__init__(...)
        self.continue_on_failure = continue_on_failure

    def report_failure(self, out, test, example, got):
        failure = doctest.DocTestFailure(test, example, got)
        if self.continue_on_failure:
            out.append(failure)
        else:
            raise failure

Doctest Item Setup Integrating Fixtures

During setup, the doctest’s globals are updated with values provided by the `doctest_namespace` fixture, enabling fixture support:

def setup(self):
    self._request._fillfixtures()
    globs = dict(getfixture=self._request.getfixturevalue)
    for name, value in self._request.getfixturevalue("doctest_namespace").items():
        globs[name] = value
    self.dtest.globs.update(globs)

Collection Decision Logic

The collector decides whether to collect a file as a doctest module or textfile:

def pytest_collect_file(file_path, parent):
    if file_path.suffix == ".py" and config.option.doctestmodules and not is_setup_or_main_py(file_path):
        return DoctestModule.from_parent(parent, path=file_path)
    elif is_doctest_text_file(config, file_path, parent):
        return DoctestTextfile.from_parent(parent, path=file_path)
    return None

Diagram

flowchart TD
    A[Start: Pytest Test Collection]
    A --> B{File Type}
    B -->|Python Module (.py)| C{--doctest-modules?}
    C -->|Yes| D[Collect DoctestModule]
    C -->|No| E[Skip Doctest Collection]
    B -->|Text File (.txt/.rst)| F{Matches doctest glob?}
    F -->|Yes| G[Collect DoctestTextfile]
    F -->|No| E
    D --> H[Find Doctests in Module]
    G --> I[Parse Doctests in Text]
    H --> J[Create DoctestItems]
    I --> J
    J --> K[Setup DoctestItem: Inject Fixtures]
    K --> L[Run Doctest with Custom Runner]
    L --> M{Failures?}
    M -->|Yes| N[Aggregate & Report Failures]
    M -->|No| O[Pass]
    N --> P[Display Detailed Diff]
    O --> P
    P --> Q[Continue or Stop Based on Config]

This flowchart visualizes the lifecycle of doctest support from collection through execution and failure reporting, highlighting key decision points and integration with fixtures and configuration.

Doctest Support enriches pytest’s versatility by enabling embedded documentation tests to participate fully in the testing workflow, benefiting from pytest’s powerful fixture system, reporting, and extensibility.