test_warnings.py

Overview

The [test_warnings.py](/projects/286/67336) file is a comprehensive test suite for validating how the pytest testing framework handles Python warnings during test execution. Its primary purpose is to ensure that warnings generated by user code, pytest internals, or test environment setup/teardown are correctly captured, displayed, filtered, grouped, and reported according to various configurations and user inputs.

The tests cover multiple warning scenarios, including:

• Warnings triggered by user code during tests and fixtures.

• Filtering warnings via command-line options, ini configuration, and pytest marks.

• Handling warnings as errors or ignoring them.

• Correct stack level reporting for warnings.

• Grouping warnings by message and summarizing them.

• Special cases such as warnings during test collection, deprecation warnings, resource warnings, and internal pytest warnings.

• Ensuring compatibility and correct precedence of multiple filtering methods.

The file extensively uses `pytester`, a pytest plugin test helper, to create test files dynamically, run pytest runs programmatically, and assert expected outcomes in output streams.

Classes, Functions, and Methods

Constants

• WARNINGS_SUMMARY_HEADER:

  WARNINGS_SUMMARY_HEADER = "warnings summary"
  

  A string constant used as a header in the warnings output section of pytest runs.

Fixture: pyfile_with_warnings

@pytest.fixture
def pyfile_with_warnings(pytester: Pytester, request: FixtureRequest) -> str:

• Purpose: Dynamically creates a minimal test file and a corresponding module that emits two warnings (UserWarning and RuntimeWarning) when a function foo() is called.

• Parameters:

  • pytester: The pytest test helper providing methods to create test files and run pytest.

  • request: Provides information about the requesting test function, used here to derive a module name.

• Returns: The path to the generated test file as a string.

• Usage Example:

  def test_example(pytester, pyfile_with_warnings):
      result = pytester.runpytest(pyfile_with_warnings)
      # assertions on result
  

• Implementation Details:

  • Inserts the test directory into sys.path.

  • Creates a test file that imports a module whose name is derived from the test function name.

  • The module defines foo() which triggers two warnings and returns 1.

Test Functions

All test functions are decorated with `pytest` decorators to set warning filters and parametrize inputs. They use `pytester` to create test files, run pytest, and check output.

test_normal_flow

Checks that warnings generated during test execution are displayed in the warnings summary section.

• Filters: Shows both UserWarning and RuntimeWarning.

• Assertions: Output contains warnings summary header, file and line info for warnings, and correct number of passed tests and warnings.

test_setup_teardown_warnings

Tests warnings emitted during fixture setup and teardown phases.

• Filters: Always show UserWarning.

• Creates: A fixture that emits warnings in setup and teardown.

• Asserts: Both setup and teardown warnings appear in output.

test_as_errors

Parametrized to check two ways of enabling warnings-as-errors: command-line `-W error` and ini configuration.

• Expectation: Warnings cause test failures.

• Asserts: Output contains error message for UserWarning and test failure.

test_ignore

Parametrized test to check ignoring warnings via command-line or ini.

• Expectation: Warnings are completely ignored and warnings summary does not appear.

• Asserts: Tests pass without warnings output.

test_unicode

Tests that warnings with Unicode message content are handled and displayed properly.

test_works_with_filterwarnings

Skipped test that ensures pre-installed warning filters don't interfere with pytest's capture.

test_filterwarnings_mark

Tests that the `@pytest.mark.filterwarnings` decorator works and takes precedence over other warning filters.

• Checks ignoring and error raising behavior on warnings via marks.

test_non_string_warning_argument

Ensures warnings with non-string arguments don't break pytest.

test_filterwarnings_mark_registration

Confirms that the `filterwarnings` mark is registered with pytest and accepted with `--strict-markers`.

test_warning_recorded_hook

Verifies that warnings are recorded and reported via the `pytest_warning_recorded` hook.

• Defines a plugin WarningCollector to capture warnings during pytest run.

• Asserts warnings from different phases (config, collection, setup, call, teardown) are collected with correct metadata.

test_collection_warnings

Checks that warnings during test collection are captured and reported.

test_mark_regex_escape

Ensures that regex special characters in filterwarnings marks are handled correctly without escaping.

test_hide_pytest_internal_warnings

Tests filtering internal pytest warnings (`PytestWarning`) by various methods (none, ini, cmdline).

test_option_precedence_cmdline_over_ini

Confirms that command-line warning filters override ini configurations.

test_option_precedence_mark

Confirms that warning filters applied via marks always take precedence over ini or cmdline.

Class: TestDeprecationWarningsByDefault

Tests the default behavior of deprecation warnings during pytest runs.

• Ensures deprecation warnings are shown by default, even with customized filters.

• Tests hiding deprecation warnings via ini, mark, cmdline, or system environment.

• Tests error handling for invalid filter regex patterns.

Class: TestAssertionWarnings

Tests that pytest emits a warning when assertions are always true due to tuple parentheses misuse.

• Provides a static helper assert_result_warns to check warning messages.

Other Tests of Note

• test_warnings_checker_twice: Tests that pytest.warns context manager can be used multiple times.

• test_group_warnings_by_message and test_group_warnings_by_message_summary: Test grouping warnings by message and summarizing counts.

• test_pytest_configure_warning: Tests that warnings during pytest_configure hook are reported properly.

• TestStackLevel: Several tests verifying that warnings point to correct source code locations, especially for conftest.py, plugins, and marks.

• test_warning_on_testpaths_not_found: Warns when testpaths is configured but no files found.

• test_resource_warning: Tests resource warnings (e.g., unclosed files) and interaction with tracemalloc.

Important Implementation Details and Algorithms

• The tests use pytester to isolate testing of warning handling by creating test files and running pytest in subprocesses or in-process.

• Warning filters are applied via pytest marks, ini files, and command-line -W options to test precedence and combination.

• The pytest_warning_recorded hook is tested by implementing a plugin class that collects warnings for later assertion.

• Stack level tests ensure that warnings emitted by pytest internals point to user-relevant code lines, improving debugging usability.

• Tests for grouping and summary of warnings validate that pytest aggregates similar warnings to make output more concise.

• Use of pytest.mark.parametrize allows testing multiple configurations in a single test function.

• Skipped tests indicate planned future or conditional tests based on pytest version or known issues.

Interaction with Other Parts of the System

• This test suite interacts primarily with the core pytest framework, specifically its warning capture and reporting mechanisms.

• Uses _pytest.pytester.Pytester plugin to create test files dynamically and run pytest runs.

• Imports from _pytest.fixtures and pytest's own marker and warning APIs.

• Interacts with pytest configuration via ini files and command-line arguments.

• Tests how pytest plugins and hooks (e.g., pytest_warning_recorded, pytest_configure) integrate with warnings.

• Tests warnings during collection, setup, teardown, and test execution phases, verifying pytest's end-to-end handling.

• Some tests check interaction with Python's standard warnings module and environment variables like PYTHONWARNINGS and PYTHONTRACEMALLOC.

Mermaid Diagram: Class Diagram for Key Classes

classDiagram
    class TestDeprecationWarningsByDefault {
        +create_file(pytester: Pytester, mark: str) void
        +test_shown_by_default(pytester: Pytester, customize_filters: bool) void
        +test_hidden_by_ini(pytester: Pytester) void
        +test_hidden_by_mark(pytester: Pytester) void
        +test_hidden_by_cmdline(pytester: Pytester) void
        +test_hidden_by_system(pytester: Pytester, monkeypatch) void
        +test_invalid_regex_in_filterwarning(pytester: Pytester) void
    }

    class TestAssertionWarnings {
        +assert_result_warns(result, msg: str) void
        +test_tuple_warning(pytester: Pytester) void
    }

    class WarningCollector {
        +pytest_warning_recorded(warning_message, when, nodeid, location) void
    }

    class TestStackLevel {
        +test_issue4445_rewrite(pytester: Pytester, capwarn) void
        +test_issue4445_preparse(pytester: Pytester, capwarn) void
        +test_conftest_warning_captured(pytester: Pytester) void
        +test_issue4445_import_plugin(pytester: Pytester, capwarn) void
        +test_issue4445_issue5928_mark_generator(pytester: Pytester) void
    }

    TestDeprecationWarningsByDefault <|-- TestAssertionWarnings
    WarningCollector ..> pytest

Summary

[test_warnings.py](/projects/286/67336) is a vital part of pytest's test suite ensuring robust and user-friendly handling of Python warnings during testing. It verifies:

• Correct capturing and reporting of warnings across all test phases.

• Configurability and precedence of warning filters.

• Proper integration with pytest hooks and plugins.

• Usability improvements such as correct stacklevel reporting and grouping of warnings.

• Handling of corner cases, including Unicode warnings, resource warnings, and internal pytest warnings.

This file plays a critical role in maintaining pytest's reliability and clarity in warning management, which is essential for developers diagnosing test issues and maintaining test code quality.