failures.rst

Overview

This documentation file, **failures.rst**, is part of the pytest testing framework documentation. It provides comprehensive guidance on how to handle test failures effectively during test runs. The file covers various command-line options, debugging aids, and pytest features that help users detect, analyze, and respond to failing tests in an efficient manner.

The key topics include stopping test sessions after a certain number of failures, invoking interactive debugging sessions automatically on failures or at test starts, setting breakpoints in code, leveraging Python’s builtin breakpoint function, managing Python fault handling for crashes and timeouts, and warnings about unraisable exceptions in destructors or threads.

This file is intended for pytest users who want to improve their debugging experience and test failure handling, helping them identify problems faster and maintain test suite quality.

Detailed Explanations

Stopping after the first (or N) failures

**Purpose:** Allow users to abort the entire pytest run as soon as the first failure or a specified number of failures occur, to quickly focus on the immediate problems.

**Usage:**

pytest -x           # stop after the first failure
pytest --maxfail=2  # stop after two failures

• -x or --maxfail=1 immediately halts the test suite on the first failure.

• --maxfail=N stops after N failures.

This helps save time when many tests fail, allowing quick feedback on the most critical issues first.

Using pdb with pytest

**Purpose:** Integrate Python’s built-in debugger, `pdb`, to start an interactive debugging session automatically when a test fails or is interrupted.

**Usage:**

pytest --pdb                   # drop to pdb on every failure or KeyboardInterrupt
pytest -x --pdb                # drop to pdb on first failure then stop testing
pytest --pdb --maxfail=3       # drop to pdb on first three failures

**Important details:**

• On failure, exception info is stored in sys.last_value, sys.last_type, and sys.last_traceback.

• This can be used for postmortem analysis or with other debugging tools.

**Example:**

>>> import sys
>>> sys.last_traceback.tb_lineno
42
>>> sys.last_value
AssertionError('assert result == "ok"')

Dropping to pdb at the start of a test

**Purpose:** Invoke the debugger immediately as each test begins, allowing step-through debugging before any test code executes.

**Usage:**

pytest --trace

This is useful for diagnosing setup issues or complex test initialization sequences.

Setting breakpoints

**Introduced in pytest 2.4.0**

**Purpose:** Enable users to set breakpoints inside test code by calling `import pdb; pdb.set_trace()`.

**Behavior with pytest:**

• pytest disables output capture automatically for that test during the debugger session.

• Output capture of other tests remains unaffected.

• Output captured before invoking the debugger is preserved.

• On continuing from the debugger, output capture resumes as normal.

This seamless integration improves the debugging experience without disrupting the overall test output handling.

Using the builtin breakpoint() function

**Context:** Python 3.7+ introduced a builtin `breakpoint()` function for entering debuggers.

**pytest support:**

• When PYTHONBREAKPOINT is at its default, pytest uses its custom internal PDB UI instead of the system default.

• After tests finish, the system defaults back to the standard PDB UI.

• Passing --pdb uses the internal PDB UI for both breakpoint() calls and failures.

• The --pdbcls option allows specifying a custom debugger class.

This integration modernizes debugging support and aligns with Python’s evolving debugging interfaces.

Fault Handler

**Introduced in pytest 5.0**

**Purpose:** Automatically dump Python tracebacks on fatal errors like segmentation faults or after test timeouts, using the standard `faulthandler` module.

**Features:**

• Enabled by default unless disabled with -p no:faulthandler.

• Supports the faulthandler_timeout configuration option to dump all threads’ tracebacks if a test exceeds a specified duration.

• Integrated from the external pytest-faulthandler plugin with minor changes in configuration options.

This feature aids in diagnosing crashes and hangs by providing immediate traceback information.

Warning about unraisable exceptions and unhandled thread exceptions

**Introduced in pytest 6.2**

**Purpose:** Detect and warn about exceptions that cannot propagate normally, such as those in `__del__` methods or unhandled exceptions in threads.

**Details:**

• Unraisable exceptions (e.g., in destructors) and unhandled thread exceptions are normally silent but considered bugs.

• pytest issues warnings visible in the test summary to alert users.

• Plugins for these warnings are enabled by default unless disabled with -p no:unraisableexception or -p no:threadexception.

• Warnings can be filtered with pytest.mark.filterwarnings.

• Warning categories are pytest.PytestUnraisableExceptionWarning and pytest.PytestUnhandledThreadExceptionWarning.

This proactive warning system helps maintain test and application robustness by exposing subtle bugs.

Implementation Details and Algorithms

• The integration with pdb and breakpoint handling relies on pytest’s test runner hooking into failure reporting and test setup phases to trigger interactive debugging.

• The fault handler is implemented using Python’s faulthandler module, automatically activated on test start and controlled via configuration options.

• The unraisable exceptions detection hooks into Python internals that track exceptions which cannot propagate, capturing them for warning generation without interrupting test flow.

• Output capture management during debugging sessions ensures that user interaction with the debugger does not interfere with pytest’s test output consistency.

Interaction with Other System Components

• pytest Core: This file documents features implemented as plugins or core pytest options that influence the test running lifecycle, failure reporting, and debugging.

• Python Standard Library: Integrates heavily with built-in modules such as pdb, faulthandler, and sys for debugging and fault handling.

• pytest Plugins: Draws on external plugins like pytest-faulthandler (now integrated) and internal plugins for unraisable exception detection.

• Command-line Interface: All features are invoked or controlled via pytest CLI options (--pdb, --maxfail, --trace, etc.), influencing test session behavior.

• Test Output Capture: Works closely with pytest’s output capturing mechanism to manage output during debugger sessions and breakpoints.

This file acts as a user-facing guide to these integrations rather than containing the actual implementation code.

Usage Examples Summary

Stop testing after first failure:

pytest -x

Invoke debugger on each failure:

pytest --pdb

Drop into debugger at test start:

pytest --trace

Set breakpoint in code:

import pdb; pdb.set_trace()

Use Python 3.7+ breakpoint function with pytest’s internal debugger:

breakpoint()

Enable fault handler with timeout dumping:

pytest -o faulthandler_timeout=10

Visual Diagram: pytest Failure Handling Features Structure

classDiagram
    class FailureHandling {
        +stop_after_failures(maxfail: int)
        +invoke_pdb_on_failure()
        +invoke_pdb_at_test_start()
        +set_breakpoint_in_code()
        +use_builtin_breakpoint()
        +enable_fault_handler(timeout: int)
        +warn_unraisable_exceptions()
    }

    FailureHandling : -maxfail
    FailureHandling : -pdb_enabled
    FailureHandling : -trace_enabled
    FailureHandling : -fault_handler_enabled
    FailureHandling : -unraisable_exception_warnings_enabled

    FailureHandling ..> "pdb (Python Debugger)"
    FailureHandling ..> "faulthandler (Python Module)"
    FailureHandling ..> "Python Exception System"

Summary

The **failures.rst** documentation file is a comprehensive reference that enables pytest users to handle test failures proactively and efficiently. By leveraging built-in and pytest-integrated debugging tools, fault handlers, and informative warnings, users can quickly identify issues and maintain high test suite reliability. The features documented here enhance the developer experience and are tightly integrated with pytest’s core and Python’s standard debugging ecosystem.