assert.rst

Overview

The `assert.rst` file is a documentation guide explaining how to write and report assertions in tests using the `pytest` testing framework. It covers the use of Python's built-in `assert` statement enhanced by `pytest`'s introspection capabilities, how to assert exceptions and warnings, how to customize assertion failure messages, and details about pytest's assertion rewriting mechanism. This file does not contain executable code but provides essential instructions and examples for writing effective tests with assertions in pytest.

The documentation is targeted primarily at developers writing tests and aims to improve test expressiveness, readability, and debugging efficiency by leveraging pytest's advanced assertion features.

Detailed Documentation

1. Using the assert Statement in pytest

`pytest` allows the direct use of Python's built-in `assert` statement for writing test assertions. Unlike standard Python assertions, pytest rewrites these assertions during test collection to provide rich introspection and detailed failure messages.

**Example:**

def f():
    return 3

def test_function():
    assert f() == 4

If the assertion fails, pytest shows the actual returned value and expression subcomponents involved in the failure, aiding debugging.

**Assertion with message:**

assert a % 2 == 0, "value was odd, should be even"

The message is printed alongside introspection details if the assertion fails.

2. Assertions About Expected Exceptions

To assert that certain exceptions are raised, pytest provides the `pytest.raises` context manager.

**Basic usage:**

import pytest

def test_zero_division():
    with pytest.raises(ZeroDivisionError):
        1 / 0

**Accessing exception info:**

def test_recursion_depth():
    with pytest.raises(RuntimeError) as excinfo:
        def f():
            f()
        f()
    assert "maximum recursion" in str(excinfo.value)

• excinfo is an instance of pytest.ExceptionInfo with attributes .type, .value, and .traceback.

**Matching exact exception type:**

def test_foo_not_implemented():
    def foo():
        raise NotImplementedError
    with pytest.raises(RuntimeError) as excinfo:
        foo()
    assert excinfo.type is RuntimeError

Here, the test fails because `NotImplementedError` is a subclass of `RuntimeError`, but the exact type check fails.

3. Matching Exception Messages

`pytest.raises` accepts a `match` parameter, a regex pattern to match against the exception's string representation.

def myfunc():
    raise ValueError("Exception 123 raised")

def test_match():
    with pytest.raises(ValueError, match=r".* 123 .*"):
        myfunc()

• The match parameter uses re.search semantics.

• It also matches against PEP-678 __notes__ if present.

4. Assertions About Exception Groups

Supports Python’s `ExceptionGroup` and `BaseExceptionGroup` via `pytest.RaisesGroup`.

**Example:**

def test_exception_in_group():
    with pytest.RaisesGroup(ValueError):
        raise ExceptionGroup("group msg", [ValueError("value msg")])
    with pytest.RaisesGroup(ValueError, TypeError):
        raise ExceptionGroup("msg", [ValueError("foo"), TypeError("bar")])

• Supports match (string regex to match group message).

• Supports check (custom callable to validate the group).

Additional options:

• flatten_subgroups to flatten nested groups.

• allow_unwrapped to allow exceptions not wrapped in groups.

**Using `pytest.RaisesExc` for detailed exception matching inside groups:**

def test_raises_exc():
    with pytest.RaisesGroup(pytest.RaisesExc(ValueError, match="foo")):
        raise ExceptionGroup("", (ValueError("foo")))

**Manual matching example:**

exc = ValueError()
exc_group = ExceptionGroup("", [exc])
if RaisesGroup(ValueError).matches(exc_group):
    ...

Detailed failure reasons are available via `.fail_reason`.

5. Using ExceptionInfo.group_contains() for Exception Groups

`excinfo.group_contains()` checks if an exception group contains a specific exception type optionally matching a regex.

**Example:**

with pytest.raises(ExceptionGroup) as excinfo:
    raise ExceptionGroup("Group message", [RuntimeError("Exception 123 raised")])
assert excinfo.group_contains(RuntimeError, match=r".* 123 .*")
assert not excinfo.group_contains(TypeError)

• depth parameter restricts search to a specific nesting level.

**Warning:** This method is not suitable for asserting that *only* certain exceptions exist in a group. For strict matching, use `pytest.RaisesGroup`.

6. Alternate pytest.raises Form (Legacy)

`pytest.raises` can be used as a function that takes a callable and arguments:

def func(x):
    if x <= 0:
        raise ValueError("x needs to be larger than zero")

pytest.raises(ValueError, func, x=-1)

This form is supported but less preferred compared to the context manager form.

7. Using pytest.mark.xfail with raises

You can mark tests expected to fail with specific exceptions:

def f():
    raise IndexError()

@pytest.mark.xfail(raises=IndexError)
def test_f():
    f()

• Only "xfail" if the raised exception or subclass matches.

• Useful for documenting known bugs.

`pytest.RaisesGroup` can also be used here for exception groups.

8. Assertions About Expected Warnings

Use `pytest.warns` to assert expected warnings (not detailed in this file but referenced).

9. Context-Sensitive Comparisons

pytest provides enhanced assertion introspection for comparisons such as:

• Sets: shows extra/missing elements.

• Strings: shows contextual diffs.

• Sequences & dicts: shows first differing element or differing entries.

Example:

def test_set_comparison():
    set1 = set("1308")
    set2 = set("8035")
    assert set1 == set2

Failure output clearly highlights extra/missing set items.

10. Custom Assertion Explanations

Implement the hook `pytest_assertrepr_compare(op, left, right)` in `conftest.py` to provide custom failure messages.

**Example:**

# conftest.py
from test_foocompare import Foo

def pytest_assertrepr_compare(op, left, right):
    if isinstance(left, Foo) and isinstance(right, Foo) and op == "==":
        return [
            "Comparing Foo instances:",
            f"   vals: {left.val} != {right.val}",
        ]

Used as:

# test_foocompare.py
class Foo:
    def __init__(self, val):
        self.val = val
    def __eq__(self, other):
        return self.val == other.val

def test_compare():
    f1 = Foo(1)
    f2 = Foo(2)
    assert f1 == f2

11. Warning on Returning Non-None from Test Functions

If a test returns a non-`None` value (e.g., a boolean), pytest emits `PytestReturnNotNoneWarning` because pytest ignores test function return values.

**Incorrect:**

@pytest.mark.parametrize(["a", "b", "result"], [[1, 2, 5]])
def test_foo(a, b, result):
    return foo(a, b) == result  # Incorrect; test won't fail on False

**Correct:**

@pytest.mark.parametrize(["a", "b", "result"], [[1, 2, 5]])
def test_foo(a, b, result):
    assert foo(a, b) == result

12. Assertion Introspection and Rewriting

pytest rewrites test modules' assert statements to capture detailed introspection data on failures. This is done on test modules discovered by pytest.

• Supporting modules are not rewritten unless manually enabled via register_assert_rewrite.

• Rewritten modules are cached on disk as .pyc files.

• Caching can be disabled with sys.dont_write_bytecode = True.

• Asserts can be disabled from rewriting by:

  • Adding PYTEST_DONT_REWRITE in the module docstring.

  • Using pytest option --assert=plain.

For more details, see Benjamin Peterson's blog post on pytest's assertion rewriting.

Interaction with Other Parts of the System

• This documentation file is part of the pytest ecosystem, serving as a user-facing guide.

• It references other pytest components like:

  • pytest.raises and pytest.warns functions.

  • pytest.ExceptionInfo class for exception introspection.

  • pytest.RaisesGroup and pytest.RaisesExc classes for exception group handling.

  • pytest's test collection and assertion rewriting internals.

• It links to other documentation topics such as tbreportdemo for reporting demos and conftest.py for configuration hooks.

• The file guides users on how to write tests that interact with pytest’s assertion mechanisms and reporting tools.

Visual Diagram: Flowchart of Main Assertion-Related Concepts in assert.rst

flowchart TD
    A[Start Writing Test] --> B{Assertion Type?}
    B -->|Simple Condition| C[Use Python assert]
    B -->|Exception| D[Use pytest.raises]
    B -->|Exception Group| E[Use pytest.RaisesGroup]
    B -->|Warning| F[Use pytest.warns]
    C --> G[pytest assertion rewriting]
    D --> H[Access ExceptionInfo: .type, .value, .traceback]
    E --> I[Match groups with match/check params]
    F --> J[Check warning types]
    G --> K[Rich failure introspection & messages]
    H --> K
    I --> K
    J --> K
    K --> L[Run test and report results]

Summary

The `assert.rst` file provides comprehensive guidance on writing assertions in pytest tests, including:

• Writing simple assertions with assert.

• Asserting raised exceptions with pytest.raises.

• Handling Python 3.11+ exception groups with pytest.RaisesGroup.

• Matching exception messages via regex.

• Customizing assertion failure messages.

• Understanding pytest's assertion rewriting mechanism.

• Best practices for test return values.

• Integration with pytest's warning assertion and xfail features.

This documentation is essential for writing robust, readable, and informative tests using pytest's powerful assertion capabilities.