test_assertion.py

Overview

`test_assertion.py` is a comprehensive test suite for the assertion rewriting and detailed assertion failure reporting mechanisms in the `pytest` testing framework. The primary focus of this file is to verify and validate how `pytest` processes and formats assertion failures, including complex comparisons, assertion rewriting hooks, and integration with plugins and various Python data structures.

This file contains numerous test classes and functions that:

• Mock configurations to test verbosity and assertion override behaviors.

• Confirm assertion rewriting is correctly applied on import hooks and plugins.

• Test detailed diffs and explanations generated when assertions fail on various data types (lists, dicts, sets, bytes, dataclasses, attrs classes, namedtuples, etc.).

• Validate formatting and truncating of assertion failure explanations.

• Check proper handling of corner cases such as unicode, faulty __repr__(), multiprocessing exceptions, and Python interpreter optimizations.

• Ensure that assertion messages, traceback locations, and warnings behave as expected.

The file uses the `pytest` testing framework and its utilities extensively for setup, test execution, and validation of outputs.

Detailed Explanation of Key Components

Functions

mock_config(verbose: int = 0, assertion_override: int | None = None) -> Config

Creates a mock configuration object mimicking a `pytest` `Config` instance, primarily for controlling verbosity settings during tests.

• Parameters:

  • verbose (int): Default verbosity level.

  • assertion_override (int | None): Optional override for assertion verbosity.

• Returns:

  • Config instance with:

    • get_terminal_writer() returning a dummy writer that highlights code without color.

    • get_verbosity(verbosity_type: str | None) returning verbosity based on the input or raising KeyError if unsupported.

• Usage Example:

  config = mock_config(verbose=3, assertion_override=10)
  verbosity = config.get_verbosity(_Config.VERBOSITY_ASSERTIONS)  # returns 10
  

callop(op: str, left: Any, right: Any, verbose: int = 0) -> list[str] | None

Helper function to call `pytest_assertrepr_compare` with the given operator and operands, returning the assertion explanation lines.

• Parameters:

  • op (str): Comparison operator (e.g., "==").

  • left (Any): Left-hand side value.

  • right (Any): Right-hand side value.

  • verbose (int): Verbosity level for explanation detail.

• Returns:

  • List of explanation strings or None if no explanation is generated.

• Usage Example:

  diff = callop("==", [1, 2], [1, 3], verbose=1)
  

callequal(left: Any, right: Any, verbose: int = 0) -> list[str] | None

Convenience wrapper for `callop` with the `"=="` operator.

• Parameters and Returns: Same as callop but fixed operator "==".

• Usage Example:

  explanation = callequal("spam", "eggs")
  

Classes and Test Suites

The file defines multiple `Test*` classes, each grouping related tests. These are `pytest` test classes (not traditional classes with complex state), using test methods with assert statements and `pytest` fixtures.

TestMockConfig

Tests behavior of the `mock_config` function, verifying correct verbosity retrieval and error raising for unsupported verbosity types.

• Examples:

  • test_verbose_exposes_value() verifies the default verbosity.

  • test_get_assertion_override_set_custom_value() tests assertion verbosity override.

  • test_get_unsupported_type_error() ensures unsupported verbosity types raise KeyError.

TestImportHookInstallation

Tests the installation and effect of assertion rewriting import hooks in various scenarios:

• Different modes (plain vs rewrite) of assertion rewriting.

• Whether conftest.py files and plugins are correctly rewritten.

• Correct handling of plugins specified as strings or lists.

• Ensures assertion rewriting is applied early enough, including for installed plugins.

• Tests registering additional modules for assertion rewriting.

Key scenarios tested include:

• Assertion failures in conftest.py fixtures.

• Assertion rewriting in pytest internal plugins (pytester).

• Plugin autoload disabling via environment variables or CLI flags.

• Recursive and nested package plugin rewriting.

• Integration with pytest.register_assert_rewrite.

TestBinReprIntegration

Tests that the `pytest_assertrepr_compare` hook is correctly called and integrated, including capturing values passed in the hook.

TestAssert_reprcompare

Extensive tests for the `pytest_assertrepr_compare` hook and its output on comparing various data types:

• Different types and basic equality.

• Text diffs with and without verbosity.

• Handling of bytes with special diffs.

• Collections: lists, dicts, sets, namedtuples, and subclasses of collections.abc.Sequence.

• Handling of faulty __repr__ implementations.

• Unicode and mojibake handling.

• Full diff output formatting and line skipping.

• Truncation and verbosity effects on diff detail.

These tests validate that detailed diffs and explanations are generated accurately and robustly for assertion errors.

TestAssert_reprcompare_dataclass

Tests assertion comparison explanations specifically for Python `dataclasses`, including recursive dataclasses and verbosity levels.

• Validates output messages that show matching and differing attributes.

• Handles corner cases like InitVar fields and custom equality methods.

TestAssert_reprcompare_attrsclass

Similar to dataclass tests but for the `attrs` library.

• Tests recursive attribute comparisons.

• Verbosity effects.

• Handling of fields with eq=False (excluded from comparison).

• Behavior when comparing different attrs class types.

• Custom __eq__ methods and auto-detection.

TestAssert_reprcompare_namedtuple

Tests comparisons of Python `NamedTuple` instances and the generated diff explanations.

• Checks for attribute-level diffs.

• Ensures fallback to sequence comparison when types differ.

TestFormatExplanation

Tests the formatting logic for assertion failure explanations, including:

• Handling of special characters and newlines.

• Formatting of where and and clauses for variable explanations.

• Proper spacing and indentation.

TestTruncateExplanation

Tests the truncation logic applied to long assertion failure explanations to keep output manageable.

• Verifies truncation by line count and character count.

• Checks edge cases where truncation message length affects output.

• Validates truncation messages indicating number of hidden lines.

• Ensures no truncation occurs when input is short or fits limits.

• Integration with CLI options and environment variables (e.g., CI).

TestSetAssertions

Tests set-specific assertion comparisons and explanations:

• Extra items in left or right sets.

• Proper handling of superset assertions.

• Integration of pytest_assertrepr_compare with sets.

Standalone Tests

The file also contains numerous standalone test functions verifying:

• Assertion rewriting correctness in various scenarios.

• Proper warning emission when running with Python optimizations.

• Correct traceback reporting in assertion failures.

• Handling of exceptions that lack tracebacks.

• Correct representation of Unicode and binary data in assertions.

• Proper handling of assertion messages that use tuples or raise exceptions in __repr__.

• Correct behavior in multiprocessing and nested exception chains.

• Fine-grained verbosity control in assertion explanations.

• Colorized output handling for diffs.

• Correct assertion handling when multiple tests are run in different directories with different conftest.py files.

• Handling of Python 2.5 compile issues and triple-quoted strings in assertions.

Important Implementation Details and Algorithms

• Assertion Rewriting: The tests verify that pytest's import hook rewrites Python bytecode to transform assert statements into calls that produce detailed failure explanations.

• Diff Generation: The file exercises and validates the diff algorithms used to compare complex data structures, including recursive comparisons and pretty-printing.

• Truncation Logic: The truncation algorithm balances line count and character count to trim long explanations while appending a message about hidden lines.

• Custom Comparison Hooks: Use of pytest_assertrepr_compare hook to generate custom assertion failure messages for user-defined types or scenarios.

• Handling of Faulty Repr: Defensive strategies are confirmed to handle objects whose __repr__ raises exceptions or returns unhelpful outputs.

Interaction With Other System Components

• Imports and uses internal pytest modules like _pytest.assertion.plugin, _pytest.assertion.util, and _pytest.assertion.truncate.

• Relies on pytest's core test running framework (pytester fixture) to create test files, run tests, and verify output.

• Uses pytest configuration mechanisms (Config) to test how verbosity and assertion options affect output.

• Interacts with plugins and conftest files to ensure assertion rewriting applies correctly.

• Uses attr and Python dataclasses modules to test integration with those data modeling libraries.

Usage Examples

These are mostly covered by the test methods themselves, for example:

def test_text_diff(self):
    assert callequal("spam", "eggs") == [
        "'spam' == 'eggs'",
        "",
        "- eggs",
        "+ spam",
    ]

This tests that when comparing two strings `"spam"` and `"eggs"`, the assertion explanation includes a line-by-line diff of the text.

Mermaid Diagram: Class Diagram of Key Test Classes

classDiagram
    class TestMockConfig {
        +test_verbose_exposes_value()
        +test_get_assertion_override_not_set_verbose_value()
        +test_get_assertion_override_set_custom_value()
        +test_get_unsupported_type_error()
    }

    class TestImportHookInstallation {
        +test_conftest_assertion_rewrite(pytester, initial_conftest, mode)
        +test_rewrite_assertions_pytester_plugin(pytester)
        +test_pytest_plugins_rewrite(pytester, mode)
        +test_pytest_plugins_rewrite_module_names(pytester, mode)
        +test_pytest_plugins_rewrite_module_names_correctly(pytester)
        +test_installed_plugin_rewrite(pytester, mode, monkeypatch, disable_plugin_autoload, explicit_specify)
        +test_rewrite_ast(pytester)
        +test_register_assert_rewrite_checks_types()
    }

    class TestAssert_reprcompare {
        +test_different_types()
        +test_summary()
        +test_text_diff()
        +test_bytes_diff_normal()
        +test_list()
        +test_iterable_full_diff(left, right, expected)
        +test_dict_wrap()
        +test_list_wrap_for_multiple_lines()
        +test_list_dont_wrap_strings()
        +test_dict()
        +test_dict_omitting()
        +test_repr_no_exc()
        +test_unicode()
        +test_nonascii_text()
        +test_format_nonascii_explanation()
        +test_nfc_nfd_same_string()
    }

    class TestAssert_reprcompare_dataclass {
        +test_dataclasses(pytester)
        +test_recursive_dataclasses(pytester)
        +test_recursive_dataclasses_verbose(pytester)
        +test_dataclasses_verbose(pytester)
        +test_dataclasses_with_attribute_comparison_off(pytester)
    }

    class TestAssert_reprcompare_attrsclass {
        +test_attrs()
        +test_attrs_recursive()
        +test_attrs_recursive_verbose()
        +test_attrs_verbose()
        +test_attrs_with_attribute_comparison_off()
        +test_comparing_two_different_attrs_classes()
        +test_attrs_with_custom_eq()
    }

    class TestAssert_reprcompare_namedtuple {
        +test_namedtuple()
        +test_comparing_two_different_namedtuple()
    }

    class TestFormatExplanation {
        +test_special_chars_full(pytester)
        +test_fmt_simple()
        +test_fmt_where()
        +test_fmt_and()
        +test_fmt_where_nested()
        +test_fmt_newline()
        +test_fmt_newline_escaped()
        +test_fmt_newline_before_where()
        +test_fmt_multi_newline_before_where()
    }

    class TestTruncateExplanation {
        +test_doesnt_truncate_when_input_is_empty_list()
        +test_truncates_at_8_lines_when_given_list_of_empty_strings()
        +test_truncates_at_8_lines_when_first_8_lines_are_LT_max_chars()
        +test_full_output_truncated(monkeypatch, pytester)
        +test_truncation_with_ini(monkeypatch, pytester, truncation_lines, truncation_chars, expected_lines_hidden)
    }

    class TestSetAssertions {
        +test_set_extra_item(op, pytester)
        +test_set_proper_superset_equal(pytester, op)
        +test_pytest_assertrepr_compare_integration(pytester)
    }

    TestImportHookInstallation --|> TestMockConfig
    TestAssert_reprcompare --|> TestMockConfig
    TestAssert_reprcompare_dataclass --|> TestAssert_reprcompare
    TestAssert_reprcompare_attrsclass --|> TestAssert_reprcompare
    TestAssert_reprcompare_namedtuple --|> TestAssert_reprcompare

Summary

`test_assertion.py` is a crucial part of the `pytest` internal test suite that ensures the robustness, correctness, and user-friendly reporting of assertion failures enhanced by `pytest`'s assertion rewriting mechanism. It covers a wide spectrum of use cases, data types, and edge cases, providing confidence in the behavior and output of assertion-related features in `pytest`.

The file is primarily a large collection of test cases with helper functions, focusing on exercising the assertion comparison logic, formatting, and integration with plugin and configuration systems of `pytest`. It helps guarantee that assertion failures are reported in a clear, detailed, and actionable manner to end users running tests with `pytest`.

End of Documentation for test_assertion.py