Warnings and Unraisable Exception Handling

This module provides comprehensive support for capturing, recording, filtering, and reporting Python warnings and unraisable exceptions during test runs. It ensures that warnings and exceptions that are normally difficult to detect or manage are surfaced in a controlled and user-friendly manner within the pytest testing lifecycle. This enhances test diagnostics and helps maintain code quality by making deprecations, runtime warnings, and unraisable exceptions visible and actionable.

Core Concepts and Purpose

Why This Module Exists

• Warnings Management: Python warnings (e.g., DeprecationWarning, PendingDeprecationWarning, FutureWarning) provide early signals about deprecated or problematic code usage but are often ignored or hidden by default. This module integrates warnings into pytest’s testing process, capturing them reliably during test phases and allowing assertions on their occurrence.

• Unraisable Exceptions Handling: Python can encounter exceptions that occur in contexts where they cannot be raised normally (e.g., during object finalization). These "unraisable exceptions" are typically logged to stderr but are hard to capture programmatically. This module intercepts Python’s unraisable exception hook to collect and report these exceptions as warnings.

• Warning Filters and Reporting: It supports applying user-defined warning filters from configuration files, command line options, and test marks, enabling fine-grained control over which warnings are shown, ignored, or converted to errors during tests.

How the Module Works

Warning Capture and Assertion

• The module defines a context manager catch_warnings_for_item (in src/_pytest/warnings.py) which uses Python’s built-in warnings.catch_warnings to capture warnings emitted within a block of code. It applies configured warning filters from pytest ini files, command line options, and marker annotations.

• Warnings are recorded and then dispatched via the pytest hook pytest_warning_recorded, enabling plugins and reporters to consume the captured warnings.

• This context manager is integrated into key pytest hooks (pytest_runtest_protocol, pytest_collection, pytest_terminal_summary, pytest_sessionfinish, pytest_load_initial_conftests) wrapping the respective phases. This ensures warnings are captured during test collection, setup, execution, and teardown phases transparently.

• The module provides utilities to convert captured warnings into formatted strings for terminal reporting (warning_record_to_str).

• Additionally, there is a fixture recwarn (in src/_pytest/recwarn.py) that test functions can use to explicitly record warnings and make assertions on them. This fixture provides a WarningsRecorder object that records warnings and offers helper methods for querying and asserting warning occurrences.

• Helper functions like pytest.warns and pytest.deprecated_call simplify asserting that code emits expected warnings, either as context managers or by wrapping function calls.

• The WarningsChecker class verifies that the expected warnings were emitted during a test and raises failures if not. It also re-emits unmatched warnings to avoid silent drops.

Example usage of the `recwarn` fixture:

def test_myfunc(recwarn):
    import warnings
    warnings.warn("deprecated", DeprecationWarning)
    w = recwarn.pop(DeprecationWarning)
    assert "deprecated" in str(w.message)

Unraisable Exception Management

• Python 3.4+ introduced sys.unraisablehook to handle exceptions that cannot be propagated normally. This module overrides sys.unraisablehook (in src/_pytest/unraisableexception.py) with a custom hook that records unraisable exceptions during test execution.

• Unraisable exceptions are stored in a pytest-managed stash queue. The module defers processing these exceptions until control points in the test lifecycle (e.g., test setup, call, teardown phases) where it collects and emits warnings for all accumulated unraisable exceptions.

• If warnings are treated as errors (e.g., -Werror option), the module raises exceptions to fail the test run accordingly.

• The module also performs multiple garbage collection iterations (gc_collect_harder) to ensure all unraisable exceptions are collected before finishing.

• Cleanup restores the original sys.unraisablehook and clears the stash to avoid memory leaks.

Warning Filters and Reporting

• Pytest applies warning filters from multiple sources to control how warnings are handled:

  • Configuration files: filterwarnings ini options.

  • Command line: -W or --pythonwarnings options.

  • Markers: @pytest.mark.filterwarnings attached to tests to apply filters scoped to individual test items.

• These filters are parsed and applied using pytest internal helpers (apply_warning_filters, parse_warning_filter) before entering warning capture contexts.

• By default, if the user has not configured Python’s warning filters (sys.warnoptions is empty), pytest enables DeprecationWarning and PendingDeprecationWarning to always show, enhancing visibility of deprecations.

• During terminal summary and session finish phases, warnings are also captured and reported to ensure no warnings are lost.

• The module adds a filterwarnings marker to the pytest configuration, allowing users to add warning filters declaratively to tests.

Interactions with Other Parts of the System

• Hooks Integration: The module hooks into pytest’s lifecycle hooks (pytest_runtest_protocol, pytest_collection, pytest_terminal_summary, pytest_sessionfinish, and pytest_load_initial_conftests) to wrap their execution within warning capture contexts, ensuring seamless integration.

• Reporting: Captured warnings are sent through the pytest_warning_recorded hook, which reporting plugins (like the terminal reporter or JUnit XML reporter) consume to present warnings to users.

• Stash Mechanism: Uses pytest’s stash system to store unraisable exception data temporarily per test session to coordinate collection and cleanup.

• Garbage Collection: Uses explicit garbage collection calls to ensure unraisable exceptions generated during cleanup phases are collected and reported.

• Fixture System: Provides the recwarn fixture for explicit warning recording and assertion inside tests.

• Traceback and Memory Profiling: Integrates with pytest’s tracemalloc support to append memory allocation traceback information to warnings and unraisable exception messages for better diagnostics.

Important Concepts and Design Patterns

• Context Manager Wrapping: Uses context managers to transparently capture warnings during various pytest phases without requiring user intervention.

• Hook Wrapping: Implements pytest hook wrappers (@pytest.hookimpl(wrapper=True)) to inject warning capture behavior around core pytest operations.

• Stash for Shared State: Employs the stash pattern for managing stateful collections of unraisable exceptions across the test run lifecycle.

• Deferred Processing: Accumulates unraisable exceptions and processes them in batch at safe points to avoid interfering with the normal exception handling flow.

• Warning Assertion Helpers: Provides convenient decorators, context managers, and fixtures to allow test authors to assert that expected warnings are emitted, improving test expressiveness.

• Error Escalation: Converts warnings to errors when configured, ensuring that problematic warnings cause test failures.

Code References

Warning Capture Context Manager

@contextmanager
def catch_warnings_for_item(
    config: Config,
    ihook,
    when: Literal["config", "collect", "runtest"],
    item: Item | None,
    *,
    record: bool = True,
) -> Generator[None]:
    ...
    with warnings.catch_warnings(record=record) as log:
        ...
        yield
    ...
    for warning_message in log:
        ihook.pytest_warning_recorded.call_historic(...)

This is used as a wrapper around test collection, execution, and reporting phases to catch warnings.

Unraisable Exception Hook Override

def unraisable_hook(
    unraisable: sys.UnraisableHookArgs,
    /,
    *,
    append: Callable[[UnraisableMeta | BaseException], object],
) -> None:
    ...
    append(
        UnraisableMeta(
            msg=msg,
            cause_msg=cause_msg,
            exc_value=unraisable.exc_value,
        )
    )

This function replaces `sys.unraisablehook` to capture unraisable exceptions into a pytest stash queue.

WarningsRecorder Fixture Usage

@fixture
def recwarn() -> Generator[WarningsRecorder]:
    wrec = WarningsRecorder(_ispytest=True)
    with wrec:
        warnings.simplefilter("default")
        yield wrec

Test functions can use the `recwarn` fixture to record and assert warnings explicitly.

Visual Diagram: Warning and Unraisable Exception Handling Workflow

sequenceDiagram
    participant TestRunner as Pytest Test Runner
    participant WarningSys as Python Warnings System
    participant UnraisableHook as sys.unraisablehook
    participant PytestHooks as Pytest Hooks
    participant Stash as Pytest Stash Storage
    participant Reporter as Reporting Plugins

    TestRunner->>WarningSys: Enter catch_warnings_for_item context
    WarningSys-->>TestRunner: Capture warnings emitted during test phases
    TestRunner->>PytestHooks: Yield to wrapped pytest hook (e.g., runtest, collection)
    PytestHooks->>WarningSys: Emit warnings inside hook
    WarningSys-->>PytestHooks: Warnings recorded in log
    TestRunner->>Reporter: Call pytest_warning_recorded hook for each warning

    Note over UnraisableHook,Stash: Unraisable exception occurs at any time
    UnraisableHook->>Stash: Append unraisable exception metadata
    TestRunner->>Stash: On test setup/call/teardown phases
    Stash->>WarningSys: Process and convert unraisable exceptions to warnings
    WarningSys->>Reporter: Emit warnings for unraisable exceptions

    TestRunner->>WarningSys: Exit catch_warnings_for_item context
    WarningSys-->>TestRunner: Restore original warning filters

This diagram illustrates the flow of warning capture during test execution and the interception and processing of unraisable exceptions within pytest.

This documentation covers the fundamental mechanisms by which pytest captures, records, filters, asserts, and reports Python warnings and unraisable exceptions, integrating these into the test lifecycle to provide enhanced visibility and control over runtime warnings and exceptional conditions.