Output and Log Capture

This module provides comprehensive support for capturing output streams (`stdout`, `stderr`) and logging messages during test execution. This functionality is critical for diagnostics, debugging, and reporting in tests, enabling users to inspect what their code outputs or logs at various stages.

Core Concepts and Purpose

Output and log capture addresses the problem of reliably intercepting and managing:

• Standard output and error streams (stdout and stderr), which tests or code under test may write to.

• Logging records generated by the Python logging module during tests.

The module ensures that these outputs do not clutter the test terminal by default but are still available for inspection on failures or via fixtures. It also supports sophisticated features such as live logging, configurable log formatting, and selective capture methods.

Module Structure and Key Functionalities

The output and log capture functionality is divided into two complementary submodules:

1. Stdout/Stderr Capture (src/_pytest/capture.py):
   Provides mechanisms to capture the textual or binary output to standard streams during tests.
   Key features include:

   • Multiple capture methods (fd, sys, no, tee-sys) selectable via command line.

   • Support for capturing file descriptor output or Python-level stream writes.

   • Fixtures (capsys, capfd, and their binary variants) to provide test functions with access to captured output.

   • Context managers to temporarily suspend or disable capturing for use cases like interactive debugging.

   • Automatic capturing during test collection and each test phase (setup, call, teardown).

   • Handling of platform-specific quirks and workarounds (e.g., Windows console I/O).

2. Logging Capture and Formatting (src/_pytest/logging.py):
   Manages capturing of Python logging records during tests, including:

   • A custom LogCaptureHandler that stores LogRecord objects and formatted log text.

   • A caplog fixture to expose captured logs to tests.

   • Configurable log levels, formats, date formats, and auto-indentation for multiline logs.

   • Support for live logging (log messages displayed immediately during test execution).

   • File logging to user-specified log files.

   • Integration with pytest’s hook system to capture logs during test phases and add logs to reports.

   • Colorized log output for terminal display using ColoredLevelFormatter.

   • Context managers to temporarily adjust logging levels or apply filters during tests.

How the Module Works: Workflows and Interactions

Stdout/Stderr Capture Workflow

• Capture Method Selection: At test startup, the capture method is chosen (fd, sys, no, or tee-sys) based on command line options (--capture) or defaults.

• CaptureManager Initialization: The CaptureManager instance initializes appropriate capture classes (FDCapture, SysCapture, etc.) for stdin, stdout, and stderr.

• Global Capturing: During collection and each test phase (setup, call, teardown), capturing is started globally and suspended/resumed as needed.

• Fixture Capturing: When fixtures like capsys or capfd are requested by test functions, the CaptureManager delegates to the fixture's capturing instance, which takes precedence over global capturing.

• Reading Captured Output: Tests can read captured output via fixture methods like readouterr(), which returns named tuples of (out, err) strings or bytes.

• Suspend/Resume: Both global and fixture capturing support suspending and resuming, allowing temporary disabling of capture (e.g., for debugging or printing to terminal).

• Report Sections: Captured output is attached to test reports, allowing it to be shown on test failure or in verbose output.

Logging Capture Workflow

• LoggingPlugin Registration: Registered via pytest_configure, this plugin manages logging handlers and formatters, setting them up based on user options (--log-level, --log-format, etc.).

• LogCaptureHandler: A custom logging handler that collects log records and formatted log messages in-memory.

• caplog Fixture: Provides test functions with access to captured logging messages, records, and utilities for clearing or filtering logs.

• Live Logging: If enabled (--log-cli-level or log_cli ini), logs are streamed live to the terminal with proper formatting and colorization.

• File Logging: Logs can be redirected to files with configurable path, mode, and format.

• Test Phase Integration: Logging is captured separately for each test phase (setup, call, teardown), and logs are attached to the test reports.

• Context Managers: The fixture supports context managers to adjust log levels temporarily or to add filters dynamically during tests.

Interaction Between Capturing Files and Test Execution

• The CaptureManager (capture.py) and LoggingPlugin (logging.py) register pytest hooks to automatically start, suspend, and stop capturing around test phases.

• Both output and log capturing add their captured data as report sections associated with test items, making them available in terminal or XML reports.

• Fixtures (capsys, caplog) provide test authors with explicit control and inspection of captured output and logs.

• The live logging feature in logging.py disables output capturing temporarily to let logs through to the terminal.

Important Concepts and Unique Design Patterns

Separation of Concerns Between Output and Logging Capture

• Output capturing deals with intercepting raw stream data (stdout, stderr) at OS or Python level.

• Logging capture intercepts structured log records emitted via the logging framework.

• Both are integrated but implemented distinctly, allowing fine-grained control and independent configuration.

Layered Capture: Global vs Fixture-Based

• Global capture is active throughout test collection and execution by default.

• Fixture capture activates when tests explicitly request it (capsys or capfd), and takes precedence by suspending global capture.

• This layering allows both implicit and explicit capturing scenarios.

Use of Context Managers for Temporary State Changes

• Both capturing subsystems provide context managers to suspend/resume capturing or temporarily change log levels or filters.

• This pattern ensures minimal interference with test code and clean restoration of states.

Custom Logging Formatters and Live Logging

• Custom formatters (ColoredLevelFormatter, DatetimeFormatter, PercentStyleMultiline) enhance log readability with colors, timestamps, and multiline message support.

• Live logging uses a special stream handler that writes logs immediately to terminal, handling terminal formatting and output capture interaction.

Code Reference Highlights

Logging Capture — LogCaptureFixture and caplog Fixture

The `LogCaptureFixture` class exposes captured logs during a test, providing access to raw `LogRecord` objects, formatted text, and helper methods:

@property
def text(self) -> str:
    """The formatted log text."""
    return _remove_ansi_escape_sequences(self.handler.stream.getvalue())

@property
def records(self) -> list[logging.LogRecord]:
    """The list of log records."""
    return self.handler.records

The `caplog` fixture yields a `LogCaptureFixture` bound to the current test node:

@fixture
def caplog(request: FixtureRequest) -> Generator[LogCaptureFixture]:
    result = LogCaptureFixture(request.node, _ispytest=True)
    yield result
    result._finalize()

Output Capture — CaptureManager and Capture Fixtures

The `CaptureManager` controls global capturing and delegates to fixture capturing:

def start_global_capturing(self) -> None:
    self._global_capturing = _get_multicapture(self._method)
    self._global_capturing.start_capturing()

def suspend_global_capture(self, in_: bool = False) -> None:
    if self._global_capturing is not None:
        self._global_capturing.suspend_capturing(in_=in_)

Fixtures like `capsys` provide test functions with capture objects that can be queried to read captured output:

@fixture
def capsys(request: SubRequest) -> Generator[CaptureFixture[str]]:
    capman: CaptureManager = request.config.pluginmanager.getplugin("capturemanager")
    capture_fixture = CaptureFixture(SysCapture, request, _ispytest=True)
    capman.set_fixture(capture_fixture)
    capture_fixture._start()
    yield capture_fixture
    capture_fixture.close()
    capman.unset_fixture()

Capture Methods Abstraction

Capture can use different underlying implementations:

• FDCapture redirects OS-level file descriptors (e.g., 1 for stdout).

• SysCapture replaces Python-level sys.stdout or sys.stderr.

• NoCapture disables capturing.

• TeeCapture duplicates output to both original streams and capture buffers.

Mermaid Diagram: High-Level Flow of Output and Log Capture During Test Execution

sequenceDiagram
    participant TestRunner
    participant CaptureManager
    participant CaptureFixture
    participant LoggingPlugin
    participant LogCaptureHandler
    participant OSStdStreams
    participant PythonLogging

    TestRunner->>CaptureManager: start_global_capturing()
    CaptureManager->>OSStdStreams: redirect stdout/stderr
    CaptureManager->>CaptureFixture: activate if requested (capsys/capfd)
    TestRunner->>CaptureFixture: capture output during test phases
    TestRunner->>LoggingPlugin: configure logging handlers
    LoggingPlugin->>LogCaptureHandler: attach handler to root logger
    PythonLogging->>LogCaptureHandler: emit log records during test
    CaptureFixture->>CaptureManager: suspend/resume capturing as needed
    TestRunner->>CaptureManager: stop_global_capturing()
    CaptureManager->>OSStdStreams: restore stdout/stderr
    TestRunner->>TestRunner: add captured output and logs to reports

This documentation page explains the specific topic of output and log capture within the system, detailing its purpose, workings, and integration with the test lifecycle, referencing the key code components supporting this functionality.