capture.py

Overview

The `capture.py` file implements a comprehensive per-test **stdout/stderr capturing mechanism** for pytest. Its primary role is to intercept and manage output streams (`sys.stdout`, `sys.stderr`, and optionally `sys.stdin`) during test execution, enabling tests to capture, inspect, and control their own output.

The file provides multiple capturing strategies, configurable via command-line options, including:

• File descriptor capturing (fd): Redirects low-level OS file descriptors (1 for stdout, 2 for stderr) to temporary files.

• Python-level sys capturing (sys): Redirects Python's sys.stdout and sys.stderr to in-memory buffers.

• No capturing (no): Disables capturing.

• Tee capturing (tee-sys): Captures output while simultaneously allowing it to be printed live to the console.

It manages capturing both at a **global level** during test collection and execution phases and at a **fixture level** when tests explicitly request capturing via fixtures like `capsys` or `capfd`.

Key Classes and Functions

1. pytest_addoption(parser: Parser) -> None

• Purpose: Adds command-line options to configure the capturing method.

• Parameters:

  • parser: pytest's argument parser.

• Behavior:

  • Adds --capture option with choices: "fd", "sys", "no", "tee-sys".

  • Adds -s as a shortcut for --capture=no.

**Example Usage**:

pytest --capture=sys
pytest -s  # disables capturing

2. _colorama_workaround() -> None

• Purpose: On Windows, imports colorama early to ensure it attaches to the correct stdio handles before capturing starts.

• Usage: Called during pytest initial conftest loading to avoid colorama-related capture issues.

3. _readline_workaround() -> None

• Purpose: Imports readline early to attach it to the correct stdio handles, addressing issues with libedit-based Python readline on macOS or other platforms.

• Usage: Called during initial pytest configuration.

4. _windowsconsoleio_workaround(stream: TextIO) -> None

• Purpose: Workaround for Windows Unicode console handling issues when duplicating file descriptors.

• Parameters:

  • stream: Typically sys.stdout or sys.stderr.

• Behavior: Re-opens stdio streams to avoid handle closing issues on Windows when redirecting file descriptors.

5. CaptureBase (abstract base class)

• Purpose: Defines the interface for capturing implementations.

• Type Parameter: AnyStr (either str or bytes).

• Key Abstract Methods:

  • __init__(fd: int)

  • start()

  • done()

  • suspend()

  • resume()

  • writeorg(data: AnyStr)

  • snap() -> AnyStr — returns captured output and resets buffer.

**Usage**: Subclasses implement specific capturing strategies.

6. NoCapture(CaptureBase[str])

• Purpose: A no-op capture class that performs no capturing.

• Behavior: Methods are implemented as empty; snap() returns empty string.

7. SysCaptureBase(CaptureBase[AnyStr])

• Purpose: Captures output by replacing Python's sys.stdout/sys.stderr/sys.stdin with in-memory buffers.

• Key Attributes:

  • _old: Original sys stream.

  • tmpfile: Capture buffer, e.g., CaptureIO or TeeCaptureIO.

  • _state: State tracking (initialized, started, suspended, done).

• Key Methods:

  • start(): Redirects sys stream to buffer.

  • done(): Restores original sys stream.

  • suspend(): Temporarily restore original stream.

  • resume(): Re-redirect to buffer.

  • writeorg(data): Write to original stream.

  • snap(): Return and reset captured content.

8. SysCapture(SysCaptureBase[str])

• Purpose: Sys-level capturing for text (str) output.

• Overrides:

  • snap(): Returns decoded captured text.

  • writeorg(): Writes text to original stream.

9. SysCaptureBinary(SysCaptureBase[bytes])

• Purpose: Sys-level capturing for binary (bytes) output.

• Overrides:

  • snap(): Returns captured bytes.

  • writeorg(): Writes bytes to original stream buffer.

10. FDCaptureBase(CaptureBase[AnyStr])

• Purpose: Captures output by redirecting OS-level file descriptors (e.g., 1 for stdout).

• Key Attributes:

  • targetfd: File descriptor to capture.

  • targetfd_save: Saved original fd.

  • tmpfile: Temporary file used for capturing.

  • syscapture: A SysCapture or NoCapture instance for Python-level capturing when necessary.

  • _state: Capture state.

• Key Methods:

  • start(): Redirects fd to tmpfile and starts syscapture.

  • done(): Restores original fd and closes tmpfile.

  • suspend(): Temporarily disables capture by restoring original fd.

  • resume(): Re-enables capture via tmpfile.

  • snap(): Reads and resets captured content.

  • writeorg(data): Writes data to original fd.

11. FDCapture(FDCaptureBase[str])

• Purpose: FD-level capturing for text output.

• Overrides:

  • snap(): Reads text from tmpfile.

  • writeorg(data: str): Writes UTF-8 encoded data to original fd.

12. FDCaptureBinary(FDCaptureBase[bytes])

• Purpose: FD-level capturing for binary output.

• Overrides:

  • snap(): Reads bytes from tmpfile.

  • writeorg(data: bytes): Writes bytes to original fd.

13. MultiCapture(Generic[AnyStr])

• Purpose: Aggregates capture objects for stdin, stdout, and stderr streams.

• Attributes:

  • in_: CaptureBase for stdin.

  • out: CaptureBase for stdout.

  • err: CaptureBase for stderr.

  • _state: Capture state.

• Key Methods:

  • start_capturing(): Starts capturing all streams.

  • suspend_capturing(in_: bool = False): Suspends capturing.

  • resume_capturing(): Resumes capturing.

  • stop_capturing(): Stops capturing and restores streams.

  • readouterr() -> CaptureResult: Returns namedtuple with out and err captured output.

  • pop_outerr_to_orig(): Flushes captured output to original streams.

14. CaptureResult(NamedTuple, Generic[AnyStr])

• Purpose: Named tuple holding captured output.

• Fields:

  • out: Captured stdout content (str or bytes).

  • err: Captured stderr content (str or bytes).

15. CaptureManager

• Purpose: Central plugin managing capturing lifecycle at global and fixture levels.

• Attributes:

  • _method: Capture method (fd, sys, no, tee-sys).

  • _global_capturing: Active global MultiCapture instance.

  • _capture_fixture: Currently active CaptureFixture.

• Key Methods:

  • start_global_capturing(): Starts global capture.

  • stop_global_capturing(): Stops global capture.

  • suspend_global_capture(in_: bool = False): Suspends global capture.

  • resume_global_capture(): Resumes global capture.

  • set_fixture(capture_fixture): Registers a CaptureFixture, ensuring mutual exclusivity.

  • unset_fixture(): Clears the current fixture.

  • activate_fixture(): Starts fixture capturing.

  • deactivate_fixture(): Stops fixture capturing.

  • suspend_fixture(): Suspends fixture capturing.

  • resume_fixture(): Resumes fixture capturing.

  • item_capture(when: str, item: Item): Context manager to capture output during test phases, attaching captured output to test reports.

  • Several pytest hook implementations (pytest_runtest_setup, pytest_runtest_call, pytest_runtest_teardown, etc.) to integrate capturing into test lifecycle.

16. CaptureFixture(Generic[AnyStr])

• Purpose: Object returned by fixtures capsys, capfd, and their binary variants, providing API to tests for accessing captured output.

• Attributes:

  • captureclass: Capture class used (SysCapture, FDCapture, etc.).

  • request: pytest's SubRequest object.

  • _capture: Active MultiCapture instance.

  • _captured_out, _captured_err: Buffered captured output.

• Key Methods:

  • _start(): Starts capturing.

  • close(): Stops capturing and stores output.

  • readouterr() -> CaptureResult: Returns and resets captured output so far.

  • _suspend(), _resume(): Suspend and resume capturing.

  • _is_started(): Checks if currently capturing.

  • disabled(): Context manager to temporarily disable capturing.

17. Fixtures

The file exposes pytest fixtures to enable capturing in tests:

• capsys: Captures text output written to sys.stdout and sys.stderr.

• capteesys: Captures text output and also passes it through to original streams (tee).

• capsysbinary: Captures binary output written to sys.stdout and sys.stderr.

• capfd: Captures text output on OS-level file descriptors 1 and 2.

• capfdbinary: Captures binary output on OS-level file descriptors 1 and 2.

All fixtures:

• Obtain the capturemanager plugin.

• Create a CaptureFixture instance with the appropriate capture class.

• Register it with the CaptureManager.

• Start capturing before yielding to the test.

• Stop capturing after the test finishes.

**Example usage in tests:**

def test_output(capsys):
    print("hello")
    captured = capsys.readouterr()
    assert captured.out == "hello\n"

Important Implementation Details and Algorithms

• Multiple Capture Strategies: The system supports capturing at both the Python stream level and the OS file descriptor level, chosen via the --capture option.

• State Machine for Capturing: Capture classes maintain internal _state to ensure correct transitions between initialized, started, suspended, and done. Operations assert the current state to prevent misuse.

• Suspend/Resume Semantics: Both global and fixture capturing support suspending and resuming capture, allowing output to be temporarily passed through without being captured, useful for debugging or interactive sessions.

• Windows-specific Workarounds: Special handling for Windows console I/O and libraries like colorama and readline to ensure capturing and output decoration work correctly.

• Dual Layer Capturing: CaptureManager coordinates global capturing active during test phases with fixture capturing requested by tests, ensuring fixtures take precedence and avoid conflicts.

• TeeCapture: Supports duplicating output both to capture buffers and original streams for live output display during tests.

• Temporary Files for FD Capture: Redirects file descriptors to temporary files for capturing low-level output, allowing capturing of output from subprocesses or C extensions.

• NamedTuple for Results: Uses CaptureResult named tuple to return captured output consistently for both text and binary captures.

Interactions with Other System Components

• Pytest Hooks: CaptureManager registers multiple pytest hooks (pytest_runtest_setup, pytest_runtest_call, pytest_runtest_teardown, etc.) to automatically start, suspend, resume, and stop capturing during test phases and collection.

• Fixtures: Provides user-facing fixtures (capsys, capfd, etc.) that interact with the capture manager, allowing tests to explicitly control and access captured output.

• Test Reporting: Captured output is attached to test reports as sections (stdout and stderr), enabling detailed failure diagnostics.

• Integration with Plugin Manager: The capture manager registers itself as a plugin and can be accessed by other plugins or fixtures.

• Platform-specific Adjustments: Detects platform (Windows vs POSIX) and adjusts capturing strategies and workarounds accordingly.

Visual Diagram: Class Structure in capture.py

classDiagram
    class CaptureBase~AnyStr~ {
        <<abstract>>
        +__init__(fd: int)
        +start()
        +done()
        +suspend()
        +resume()
        +writeorg(data: AnyStr)
        +snap() AnyStr
    }

    class NoCapture {
        +start()
        +done()
        +suspend()
        +resume()
        +writeorg(data: str)
        +snap() str
    }

    class SysCaptureBase~AnyStr~ {
        -_old: TextIO
        -tmpfile: TextIO
        -_state: str
        +start()
        +done()
        +suspend()
        +resume()
        +writeorg(data: AnyStr)
        +snap() AnyStr
    }

    class SysCapture {
        +snap() str
        +writeorg(data: str)
    }

    class SysCaptureBinary {
        +snap() bytes
        +writeorg(data: bytes)
    }

    class FDCaptureBase~AnyStr~ {
        -targetfd: int
        -targetfd_save: int
        -tmpfile: TextIO
        -syscapture: CaptureBase
        -_state: str
        +start()
        +done()
        +suspend()
        +resume()
        +writeorg(data: AnyStr)
        +snap() AnyStr
    }

    class FDCapture {
        +snap() str
        +writeorg(data: str)
    }

    class FDCaptureBinary {
        +snap() bytes
        +writeorg(data: bytes)
    }

    class MultiCapture~AnyStr~ {
        -in_: CaptureBase
        -out: CaptureBase
        -err: CaptureBase
        -_state: str
        +start_capturing()
        +stop_capturing()
        +suspend_capturing(in_: bool)
        +resume_capturing()
        +readouterr() CaptureResult
        +pop_outerr_to_orig()
    }

    class CaptureManager {
        -_method: str
        -_global_capturing: MultiCapture
        -_capture_fixture: CaptureFixture
        +start_global_capturing()
        +stop_global_capturing()
        +suspend_global_capture(in_: bool)
        +resume_global_capture()
        +set_fixture(capture_fixture: CaptureFixture)
        +unset_fixture()
        +activate_fixture()
        +deactivate_fixture()
        +suspend_fixture()
        +resume_fixture()
    }

    class CaptureFixture~AnyStr~ {
        -captureclass: type
        -request: SubRequest
        -_capture: MultiCapture
        -_captured_out: AnyStr
        -_captured_err: AnyStr
        +_start()
        +close()
        +readouterr() CaptureResult
        +_suspend()
        +_resume()
        +disabled()
    }

    CaptureBase <|-- NoCapture
    CaptureBase <|-- SysCaptureBase
    SysCaptureBase <|-- SysCapture
    SysCaptureBase <|-- SysCaptureBinary
    CaptureBase <|-- FDCaptureBase
    FDCaptureBase <|-- FDCapture
    FDCaptureBase <|-- FDCaptureBinary

Usage Examples

Using capsys Fixture in a Test

def test_print_output(capsys):
    print("hello world")
    captured = capsys.readouterr()
    assert captured.out == "hello world\n"
    assert captured.err == ""

Temporarily Disabling Capture Within a Test

def test_disable_capture(capsys):
    with capsys.disabled():
        print("this goes directly to terminal")
    captured = capsys.readouterr()
    assert "this goes directly to terminal" not in captured.out

Summary

`capture.py` is a central module in pytest that implements robust, flexible capturing of stdout/stderr streams during test runs. It supports multiple capturing backends, manages capturing state transitions, integrates with pytest lifecycle hooks, and provides user-facing fixtures to enable fine-grained control over output capture.

This functionality is crucial for enabling clean test output, debugging, and detailed test reporting.

End of Documentation for capture.py