pytester_assertions.py

Overview

`pytester_assertions.py` is a helper plugin module designed to provide assertion utilities specifically for the `pytester` testing framework plugin within the pytest ecosystem. The module contains assertion functions that verify test run outcomes, such as the number of passed, skipped, failed, or otherwise categorized test reports.

This plugin **should not be loaded independently** because it depends on the `pytest` module's import behavior and assertion rewriting mechanism. Since `pytester` itself is imported by `pytest` and cannot be subject to assertion rewriting, these assertions are separated into this helper plugin to support `pytester`'s internal testing needs.

Detailed Description of Components

Imports

• Sequence from collections.abc: Used for type hinting sequences.

• CollectReport and TestReport from _pytest.reports: These are pytest internal classes representing reports generated during test collection and test execution phases.

Functions

assertoutcome

def assertoutcome(
    outcomes: tuple[
        Sequence[TestReport],
        Sequence[CollectReport | TestReport],
        Sequence[CollectReport | TestReport],
    ],
    passed: int = 0,
    skipped: int = 0,
    failed: int = 0,
) -> None:

**Purpose:** Asserts that the actual test outcomes match the expected counts of passed, skipped, and failed tests.

**Parameters:**

• outcomes: A tuple of three sequences:

  • First sequence: Sequence[TestReport] representing passed test reports.

  • Second sequence: Sequence[CollectReport | TestReport] representing skipped test reports or collection reports.

  • Third sequence: Sequence[CollectReport | TestReport] representing failed test reports or collection reports.

• passed (int, default=0): Expected number of passed tests.

• skipped (int, default=0): Expected number of skipped tests.

• failed (int, default=0): Expected number of failed tests.

**Returns:** None. Raises an `AssertionError` if the counts do not match.

**Usage Example:**

# Suppose 'outcomes' is obtained from a pytest test run:
outcomes = (passed_reports, skipped_reports, failed_reports)

# Assert expected counts:
assertoutcome(outcomes, passed=3, skipped=1, failed=0)

**Implementation Details:**

• Extracts the actual counts from the sequences by measuring their lengths.

• Compares the actual counts against the expected values.

• Uses a direct assert statement for validation.

• The __tracebackhide__ = True directive is used to hide this assertion from traceback output in pytest, keeping test failure reports clean.

assert_outcomes

def assert_outcomes(
    outcomes: dict[str, int],
    passed: int = 0,
    skipped: int = 0,
    failed: int = 0,
    errors: int = 0,
    xpassed: int = 0,
    xfailed: int = 0,
    warnings: int | None = None,
    deselected: int | None = None,
) -> None:

**Purpose:** Asserts that the given dictionary of test outcome counts matches the expected count values across various categories, including passed, skipped, failed, errors, expected failures, unexpected passes, warnings, and deselected tests.

**Parameters:**

• outcomes (dict[str, int]): A dictionary mapping outcome categories to their counts, typically parsed from test run summary data.

• passed, skipped, failed, errors, xpassed, xfailed (int, default=0): Expected counts for each respective category:

  • passed: Number of passed tests.

  • skipped: Number of skipped tests.

  • failed: Number of failed tests.

  • errors: Number of errors during test runs.

  • xpassed: Number of tests that were expected to fail but passed.

  • xfailed: Number of tests that failed as expected.

• warnings (int | None, default=None): Expected number of warnings. If None, warnings are not checked.

• deselected (int | None, default=None): Expected number of deselected tests. If None, deselected are not checked.

**Returns:** None. Raises an `AssertionError` if any of the expected counts do not match the actual counts.

**Usage Example:**

# outcomes dictionary obtained from a pytest test run summary:
outcomes = {
    "passed": 5,
    "failed": 1,
    "skipped": 2,
    "errors": 0,
    "xpassed": 0,
    "xfailed": 1,
    "warnings": 0,
}

# Assert expected test outcome counts:
assert_outcomes(outcomes, passed=5, failed=1, skipped=2, xfailed=1, warnings=0)

**Implementation Details:**

• Extracts outcome counts from the provided dictionary, defaulting missing keys to zero.

• Compares actual counts with expected counts for all categories.

• Only checks warnings and deselected counts if these parameters are explicitly provided (not None).

• Uses a direct assert statement for validation.

• Includes __tracebackhide__ = True to suppress the assertion frame in pytest tracebacks.

Important Implementation Details

• The module relies on Python's built-in assert statements for validation, which integrates cleanly with pytest's assertion rewriting and reporting mechanisms.

• The __tracebackhide__ attribute is used to improve the readability of test failures by hiding internal assertion frames.

• The split of assertion logic into this helper plugin is necessary because pytester itself cannot be assertion rewritten due to import order constraints.

• The functions work primarily by comparing counts of test reports or outcome summaries, not by analyzing individual test data, making them lightweight and focused on summary verification.

Interaction with Other Parts of the System

• pytester: This module is tightly coupled with pytester, a plugin used to test pytest plugins and pytest itself. The assertion functions here are used by pytester to verify that test runs produce the expected results.

• pytest: Since the module relies on pytest internal report objects (TestReport, CollectReport), it depends on pytest's internal APIs. It is designed to integrate with pytest's test reporting and assertion mechanisms.

• _pytest.reports: The module imports internal pytest report classes representing collected and executed test outcomes.

• These assertions are used in test scenarios where pytester runs tests and collects outcomes, then asserts expected results in terms of passes, fails, skips, and other categories.

Visual Diagram

flowchart TD
    A[pytester_assertions.py]

    A --> B[assertoutcome]
    B --> C[outcomes: tuple of sequences]
    B --> D[expected counts: passed, skipped, failed]
    B --> E[assert actual counts == expected]

    A --> F[assert_outcomes]
    F --> G[outcomes: dict of outcome counts]
    F --> H[expected counts: passed, skipped, failed, errors, xpassed, xfailed, (optional) warnings, deselected]
    F --> I[assert actual counts == expected]

    style A fill:#f9f,stroke:#333,stroke-width:2px
    style B fill:#bbf,stroke:#333,stroke-width:1px
    style F fill:#bbf,stroke:#333,stroke-width:1px

Summary

`pytester_assertions.py` is a concise and specialized helper module providing assertion functions tailored for validating pytest test run outcomes in the `pytester` plugin context. It encapsulates logic to compare actual test results with expected counts across multiple categories, improving test reliability and clarity. Its careful design ensures compatibility with pytest's assertion rewriting and reporting features while segregating assertion logic from `pytester`'s core to manage import constraints.