capture-warnings.rst

Overview

This file provides comprehensive documentation on how to capture, control, and assert Python warnings during test execution using the pytest testing framework. It explains pytest's built-in support for warning capture starting from version 3.1, describes mechanisms to filter and handle warnings, and demonstrates how to write tests that assert the presence or absence of warnings. The document also covers configuration options, usage of pytest marks for warnings, and advanced use cases such as recording warnings and customizing failure messages.

This documentation is intended for developers writing tests with pytest who want to manage warning output effectively and ensure that their code emits or suppresses warnings as expected.

Detailed Explanations

Capturing Warnings Automatically

• pytest automatically captures warnings emitted during test execution and displays a summary at the end of the test session.

• Example:

  import warnings
  
  def api_v1():
      warnings.warn(UserWarning("api v1, should use functions from v2"))
      return 1
  
  def test_one():
      assert api_v1() == 1
  

• When running pytest on this test, the warning is shown in a summary after tests complete.

Controlling Warnings

pytest provides a `-W` command-line option (similar to Python's own `-W` flag) to control how warnings are handled during tests:

• You can ignore, display, or turn warnings into errors.

• Example to treat all UserWarnings as errors:

  pytest -q test_show_warnings.py -W error::UserWarning
  

• You can also configure warning filters in pytest.ini or pyproject.toml using the filterwarnings option:

  # pytest.ini
  [pytest]
  filterwarnings =
      error
      ignore::UserWarning
      ignore:function ham\(\) is deprecated:DeprecationWarning
  

• Last matching filter in the list takes precedence.

@pytest.mark.filterwarnings

• Allows applying warning filters to specific tests, classes, or modules.

• Example:

  import pytest
  import warnings
  
  def api_v1():
      warnings.warn(UserWarning("api v1, should use functions from v2"))
      return 1
  
  @pytest.mark.filterwarnings("ignore:api v1")
  def test_one():
      assert api_v1() == 1
  

• Multiple filters can be stacked using multiple decorators.

• Decorator order matters: filters from earlier decorators take precedence.

• Filters applied this way override command line and ini option filters.

• Can also apply to entire modules using the pytestmark global variable.

Disabling Warning Capture

• Use --disable-warnings CLI option to suppress the warnings summary.

• Disable the warnings plugin entirely with -p no:warnings CLI option or in pytest.ini:

  [pytest]
  addopts = -p no:warnings
  

Handling DeprecationWarnings

• By default, pytest shows DeprecationWarning and PendingDeprecationWarning warnings from user and third-party code.

• If warnings are captured explicitly with pytest.warns, pytest.deprecated_call, or recwarn fixture, these warnings are suppressed from summary.

• Filters can be used to ignore specific deprecation warnings from third-party code.

Asserting Warnings with pytest.warns

• The pytest.warns context manager asserts that a certain warning is raised.

  import warnings
  import pytest
  
  def test_warning():
      with pytest.warns(UserWarning):
          warnings.warn("my warning", UserWarning)
  

• Supports a match argument to check warning message content (supports regex).

• Can be called as a function to check a callable raises a warning:

  pytest.warns(UserWarning, func, *args, **kwargs)
  

• Returns a list-like record of captured warnings.WarningMessage objects:

  with pytest.warns(RuntimeWarning) as record:
      warnings.warn("another warning", RuntimeWarning)
  assert len(record) == 1
  assert record[0].message.args[0] == "another warning"
  

Recording Warnings with recwarn Fixture

• The recwarn pytest fixture records warnings during a test function's execution.

• Example:

  import warnings
  
  def test_hello(recwarn):
      warnings.warn("hello", UserWarning)
      assert len(recwarn) == 1
      w = recwarn.pop(UserWarning)
      assert issubclass(w.category, UserWarning)
      assert str(w.message) == "hello"
      assert w.filename
      assert w.lineno
  

• Both recwarn and pytest.warns return a WarningsRecorder instance.

Additional Use Cases for Warnings in Tests

• Assert at least one warning from multiple types is raised:

  with pytest.warns((RuntimeWarning, UserWarning)):
      ...
  

• Assert only certain warnings are raised using recwarn and checking length.

• Assert no warnings are raised by converting warnings to errors temporarily.

• Suppress warnings within a code block using warnings.catch_warnings().

Custom Failure Messages

• You can customize test failure messages if no warnings are issued:

  def test():
      with pytest.warns(Warning) as record:
          f()
          if not record:
              pytest.fail("Expected a warning!")
  

Internal pytest Warnings

• pytest may issue its own warnings for improper usage or deprecated features.

• Example: warning if a test class defines __init__ constructor.

• These warnings can be filtered similarly to user warnings.

Resource Warnings and tracemalloc

• When tracemalloc is enabled (e.g., via PYTHONTRACEMALLOC environment variable), pytest provides additional source information for ResourceWarnings.

• Helps track down resource leaks with stack frame details.

Important Implementation Details

• pytest's warning capture relies on Python's standard warnings module but enhances output formatting and filtering.

• Warning filters configured via command line or ini options are translated into Python warning filters but differ slightly in pattern matching behavior.

• The @pytest.mark.filterwarnings decorator uses a reversed precedence order compared to Python's warnings.filterwarnings due to decorator evaluation order.

• The recwarn fixture resets warning filters after each test to avoid leaking global state.

• pytest suppresses deprecation warnings from captured warnings to avoid duplicate reporting.

• Internal pytest warnings are generated by the pytest core to warn users about deprecated or incorrect test definitions.

Interactions with Other Parts of the System

• This file documents features implemented by the pytest core warnings plugin.

• Interacts closely with Python's warnings module and the test collection/execution phases of pytest.

• Works in conjunction with pytest configuration files (pytest.ini, pyproject.toml) for filter settings.

• Integrates with other pytest features such as fixtures (recwarn) and markers (@pytest.mark.filterwarnings).

• Affects test reporting output by adding warnings summary sections.

• Can be disabled or modified by pytest command-line options or plugin management.

Visual Diagram

The following Mermaid class diagram illustrates the key concepts and entities related to warnings in pytest as documented here, focusing on the main interfaces and interactions for capturing and asserting warnings.

classDiagram
    class pytest {
        +warns(expected_warning, func=None, *args, **kwargs)
        +deprecated_call()
        +mark.filterwarnings(filter_str)
    }

    class WarningsRecorder {
        +__len__()
        +__getitem__(index)
        +__iter__()
        +pop(warning_category)
    }

    class recwarn {
        +__len__()
        +pop(warning_category)
        +__iter__()
    }

    class warnings {
        +warn(message, category)
        +simplefilter(action)
        +catch_warnings()
    }

    pytest ..> WarningsRecorder : returns instance
    pytest ..> recwarn : fixture instance
    WarningsRecorder <.. recwarn : similar interface
    pytest --> warnings : uses internally
    recwarn --> warnings : records warnings
    pytest.warns "context manager" --> WarningsRecorder : captures warnings

Usage Examples

Basic warning capture with pytest

import warnings

def api_v1():
    warnings.warn(UserWarning("api v1 deprecated"))
    return 1

def test_api():
    assert api_v1() == 1

Treat warnings as errors via CLI

pytest -W error::UserWarning test_module.py

Using @pytest.mark.filterwarnings to ignore warnings on a test

import pytest
import warnings

def api_v1():
    warnings.warn(UserWarning("deprecated API"))
    return 1

@pytest.mark.filterwarnings("ignore:deprecated API")
def test_api():
    assert api_v1() == 1

Asserting warning raised with pytest.warns

import warnings
import pytest

def test_warn():
    with pytest.warns(UserWarning, match="my warning"):
        warnings.warn("my warning", UserWarning)

Recording warnings with recwarn fixture

def test_recwarn(recwarn):
    import warnings
    warnings.warn("hello", UserWarning)
    w = recwarn.pop(UserWarning)
    assert "hello" in str(w.message)

Summary

This documentation file extensively covers how pytest captures, filters, controls, and asserts warnings during test runs. It provides both usage instructions and configuration guidance, enabling users to write robust tests that handle warnings appropriately. The interactions with Python's native warnings system, pytest's configuration, and decorators/fixtures are clearly explained, making this a valuable resource for pytest users concerned with warning management.

*End of documentation for `capture-warnings.rst`*