skipping.py

Overview

The [skipping.py](/projects/286/67623) module is a core part of pytest's test execution framework that provides support for **skipping tests** and marking **expected failures (xfail)**. It enables conditional test skipping and expected failure handling based on dynamically evaluated conditions or static markers. This allows test authors to declaratively specify when a test should be skipped or considered an expected failure, improving test suite flexibility, reporting clarity, and robustness.

Key capabilities include:

• Defining and evaluating skip, skipif, and xfail markers on test items.

• Conditional execution control to skip or xfail tests before or during runtime.

• Integration with pytest’s command-line options (--runxfail) and ini configuration (xfail_strict).

• Hook implementations that modify test setup, execution, and reporting phases to respect skipping and xfail logic.

• Safe evaluation of string-based conditions in a controlled namespace.

• Stashing of evaluation results for efficient reuse during the test lifecycle.

• Reporting test statuses reflecting skipped, xfail, and xpass outcomes.

Classes, Functions, and Methods

1. pytest_addoption(parser: Parser) -> None

**Purpose:** Adds command-line options and ini-file configuration related to skipping and xfail functionality.

**Details:**

• Adds the --runxfail option: if set, xfail tests run normally and their results are reported as if not marked.

• Adds the xfail_strict ini option: sets the default strictness behavior of xfail markers.

**Usage Example:**

pytest --runxfail

or in `pytest.ini`:

[pytest]
xfail_strict = true

2. pytest_configure(config: Config) -> None

**Purpose:** Performs initial configuration hooks for skip and xfail support.

**Details:**

• If --runxfail is enabled, monkeypatches pytest.xfail to a no-op to prevent actual skipping.

• Registers marker documentation strings describing skip, skipif, and xfail markers for users.

3. evaluate_condition(item: Item, mark: Mark, condition: object) -> tuple[bool, str]

**Purpose:** Evaluates a single condition from a `skipif` or `xfail` marker to determine if it triggers skipping/xfail.

**Parameters:**

• item: The pytest test item being evaluated.

• mark: The marker object (skipif or xfail) containing metadata.

• condition: Either a string expression or a boolean value representing the condition.

**Returns:** A tuple `(result: bool, reason: str)` where `result` indicates if the condition is met, and `reason` provides the reason for skip/xfail.

**Implementation Details:**

• If condition is a string, compiles and evaluates it in a restricted namespace including os, sys, platform, and config.

• If condition is boolean, it is cast to bool().

• Errors during evaluation raise a pytest failure with a detailed message.

• Ensures a meaningful reason string is provided, otherwise fails with an error.

**Example Usage:**

result, reason = evaluate_condition(item, mark, "sys.platform == 'win32'")
if result:
    # condition triggered

4. @dataclasses.dataclass(frozen=True) class Skip

**Purpose:** Represents the result of evaluating skip markers on a test item.

**Attributes:**

• reason: str — Reason why the test was skipped (defaults to "unconditional skip").

5. evaluate_skip_marks(item: Item) -> Skip | None

**Purpose:** Evaluates all `skip` and `skipif` markers on the given test item to decide if the test should be skipped.

**Returns:**

• Skip instance if any skip condition is met.

• None if the test should not be skipped.

**Implementation Details:**

• Iterates over all skipif markers on the item.

• For each, extracts conditions either from positional args or a condition keyword.

• Evaluates each condition; if any evaluates to true, returns Skip with reason.

• If no skipif triggers, checks for unconditional skip markers and returns Skip.

• Raises a friendly error if skip marker is used incorrectly (e.g., meant to be skipif).

6. @dataclasses.dataclass(frozen=True) class Xfail

**Purpose:** Represents the result of evaluating xfail markers on a test item.

**Attributes:**

• reason: str — Reason for the expected failure.

• run: bool — Whether to run the test despite xfail.

• strict: bool — Whether unexpected passes should cause test failure.

• raises: Optional[Exception or tuple or AbstractRaises] — Expected exception(s) that justify the failure.

7. evaluate_xfail_marks(item: Item) -> Xfail | None

**Purpose:** Evaluates all `xfail` markers on the test item and returns an `Xfail` instance if any conditions are met.

**Returns:**

• Xfail instance if the test should be considered an expected failure.

• None if no xfail condition applies.

**Implementation Details:**

• Iterates over xfail markers, extracting parameters (run, strict, raises, and conditions).

• Evaluates conditions similarly to skip conditions.

• Returns Xfail if any condition matches or if no condition given (unconditional xfail).

8. xfailed_key = StashKey[Optional[Xfail]]()

**Purpose:** A unique key used to stash the result of xfail evaluation inside a test item's stash dictionary, enabling efficient reuse during test phases.

9. pytest_runtest_setup(item: Item) -> None

**Purpose:** Pytest hook called before running a test, responsible for applying skip and xfail logic.

**Behavior:**

• Calls evaluate_skip_marks and raises skip.Exception if skip conditions met, aborting the test.

• Evaluates xfail marks and stores result in the item's stash.

• If xfail applies with run=False and --runxfail not enabled, calls xfail() to skip running the test.

10. pytest_runtest_call(item: Item) -> Generator[None]

**Purpose:** Wraps the test call phase, managing xfail behavior.

**Behavior:**

• Ensures the xfail evaluation is cached in the stash.

• If xfail with run=False and not overridden by --runxfail, calls xfail() to avoid running the test.

• Yields control to allow the test function to execute.

• After execution, refreshes the xfail evaluation in case it changed dynamically.

11. pytest_runtest_makereport(item: Item, call: CallInfo[None]) -> Generator[None, TestReport, TestReport]

**Purpose:** Wraps the test reporting phase to modify reports based on skip and xfail status.

**Behavior:**

• Retrieves xfail info from stash.

• If --runxfail is enabled, leaves report unchanged.

• If test raised an xfail.Exception, marks report as skipped with wasxfail message.

• Otherwise, if test failed or passed, interprets xfail conditions:

  • If failure matches expected exceptions (raises), marks as skipped and wasxfail.

  • If failure does not match expected exceptions, marks as failed.

  • If test passed but strict is true, marks as failed with [XPASS(strict)] message.

  • If test passed and not strict, marks as passed with wasxfail.

• Returns the possibly modified report.

12. pytest_report_teststatus(report: BaseReport) -> tuple[str, str, str] | None

**Purpose:** Customizes terminal test status reporting for xfail/skipped tests.

**Returns:**

• For tests marked wasxfail and skipped: ("xfailed", "x", "XFAIL")

• For tests marked wasxfail and passed: ("xpassed", "X", "XPASS")

• Otherwise: None (no change)

Important Implementation Details and Algorithms

• Condition Evaluation:
  String-based conditions are securely evaluated with eval() using a controlled global namespace that includes standard modules (os, sys, platform) and the test item's global namespace. This allows powerful, expressive conditional skipping/xfail logic without arbitrary code execution risk.

• Marker Parameter Handling:
  Conditions can be passed as positional arguments or keyword argument condition. The module supports legacy and newer marker usage.

• Stashing:
  The use of StashKey to store xfail evaluation results in the test item's stash dictionary avoids redundant condition evaluations across test setup, call, and reporting phases, improving efficiency.

• Exception-Based Control Flow:
  Skipping and xfail logic is enforced by raising special exceptions (skip.Exception and xfail.Exception) to interrupt or modify the test execution flow seamlessly.

• Hook Wrapping:
  The test phases (runtest_setup, runtest_call, runtest_makereport) are wrapped or prioritized (tryfirst) to ensure skip/xfail evaluation runs early and can influence or override default behavior.

• Report Mutation:
  The module carefully mutates TestReport instances to reflect xfail/skipped test results, including setting attributes like wasxfail, outcome, and longrepr for accurate reporting.

Interaction with Other System Components

• pytest.nodes.Item:
  The test item object represents individual tests during collection and execution. Markers are attached to items and evaluated here. The stash mechanism used to store xfail state is part of the item.

• pytest.config.Config:
  Provides access to command line options (e.g., --runxfail) and ini configuration (xfail_strict), used to control behavior.

• pytest.outcomes:
  Contains the special exceptions (skip.Exception, xfail.Exception) that signal skipping or expected failure.

• pytest.mark.structures.Mark:
  Marker objects attached to items represent the skip, skipif, and xfail annotations.

• pytest.reports.TestReport:
  The test report object is modified in pytest_runtest_makereport to reflect the test’s final outcome considering skip/xfail logic.

• pytest.stash.StashKey:
  Used to create a unique key for storing xfail evaluations in the test item’s stash dictionary.

The module hooks into pytest’s lifecycle via:

• pytest_addoption and pytest_configure for setup.

• pytest_runtest_setup to evaluate skip/xfail before test execution.

• pytest_runtest_call to conditionally run or skip tests.

• pytest_runtest_makereport to modify reporting outcomes.

• pytest_report_teststatus to customize terminal output.

Mermaid Class Diagram

classDiagram
    class Skip {
        +reason: str
    }
    class Xfail {
        +reason: str
        +run: bool
        +strict: bool
        +raises: type or tuple or AbstractRaises or None
    }
    class StashKey~T~ {
        <<generic>>
    }
    class Mark {
        +name: str
        +args: tuple
        +kwargs: dict
    }
    class Item {
        +config: Config
        +stash: dict
        +iter_markers(name: str) Mark[]
    }
    class CallInfo~T~ {
        +excinfo: ExceptionInfo | None
        +when: str
    }
    class TestReport {
        +wasxfail: Optional[str]
        +outcome: str
        +longrepr: Optional[str]
    }

    Skip ..> Mark : uses
    Xfail ..> Mark : uses
    Item ..> Mark : has markers
    Item ..> StashKey~Xfail~ : uses for xfail cache
    CallInfo ..> Exception : may contain
    pytest_runtest_makereport ..> TestReport : modifies

Usage Summary

# Example: Mark a test to skip on Windows platforms
@pytest.mark.skipif(sys.platform == "win32", reason="Does not run on Windows")
def test_windows_specific():
    ...

# Example: Mark a test as expected to fail if an environment variable is set
@pytest.mark.xfail(os.getenv("BUGGY_FEATURE") == "1", reason="Known bug #123", strict=True)
def test_buggy_feature():
    ...

# During test setup, skipping and xfail markers are evaluated automatically
# by pytest via the hooks defined in this module.

Summary

The [skipping.py](/projects/286/67623) module is essential for pytest’s flexible test skipping and expected failure management. It ensures that tests can be conditionally excluded or expected to fail without disrupting test suite execution and reporting. By providing a robust evaluation mechanism, integration with pytest’s lifecycle, and clear feedback to users, it significantly enhances test suite maintainability and clarity.

End of skipping.py documentation.