simple.rst

Overview

This file provides a comprehensive set of **basic patterns and examples** for using and extending the `pytest` testing framework. It serves as a tutorial and reference guide demonstrating common and advanced pytest usage scenarios, including command line option customization, test parametrization based on CLI input, test skipping control, incremental testing, fixture usage, test reporting customization, profiling, and integration with packaging tools.

The content is primarily intended for pytest users and plugin authors who want to understand practical ways to extend and control pytest behavior in projects, or manage test execution more effectively.

The file is organized into multiple example-driven sections, each focusing on a specific pytest feature or pattern, illustrated with code samples and usage explanations.

Detailed Explanations of Key Concepts and Code Examples

1. Changing Command Line Options Defaults

By default, pytest accepts command line options at runtime. To avoid repetitive typing of common options, you can specify defaults in a configuration file `pytest.ini` or via the environment variable `PYTEST_ADDOPTS`.

• pytest.ini example:

  [pytest]
  addopts = -ra -q
  

• Environment variable example:

  export PYTEST_ADDOPTS="-v"
  

• Command line composition:

  When pytest runs, the final command line is:

  <pytest.ini:addopts> $PYTEST_ADDOPTS <extra command-line arguments>
  

  Options specified later override earlier ones in case of conflicts.

2. Passing Different Values to Tests Based on Command Line Options

You can define custom command line options and make them available to test functions via fixtures.

• conftest.py example adding --cmdopt option:

  import pytest
  
  def pytest_addoption(parser):
      parser.addoption(
          "--cmdopt",
          action="store",
          default="type1",
          help="my option: type1 or type2",
          choices=("type1", "type2"),
      )
  
  @pytest.fixture
  def cmdopt(request):
      return request.config.getoption("--cmdopt")
  

• Test file test_sample.py using the fixture:

  def test_answer(cmdopt):
      if cmdopt == "type1":
          print("first")
      elif cmdopt == "type2":
          print("second")
      assert 0  # to see what was printed
  

• Usage:

  $ pytest -q test_sample.py
  $ pytest -q --cmdopt=type2
  

• Validation with custom type checker and detailed error messages:

  You can define a type function that raises pytest.UsageError to validate or parse options.

3. Dynamically Adding Command Line Options

Plugins or external modules can dynamically modify pytest command line arguments before pytest processes them.

• Example plugin snippet:

  import sys
  
  def pytest_load_initial_conftests(args):
      if "xdist" in sys.modules:
          import multiprocessing
          num = max(multiprocessing.cpu_count() // 2, 1)
          args[:] = ["-n", str(num)] + args
  

This example automatically adds parallel execution options if the `pytest-xdist` plugin is available.

4. Controlling Skipping of Tests Based on Command Line Options

You can add a CLI option to control whether tests marked with a custom marker (`slow`) are run or skipped.

• conftest.py example:

  import pytest
  
  def pytest_addoption(parser):
      parser.addoption(
          "--runslow", action="store_true", default=False, help="run slow tests"
      )
  
  def pytest_configure(config):
      config.addinivalue_line("markers", "slow: mark test as slow to run")
  
  def pytest_collection_modifyitems(config, items):
      if config.getoption("--runslow"):
          return
      skip_slow = pytest.mark.skip(reason="need --runslow option to run")
      for item in items:
          if "slow" in item.keywords:
              item.add_marker(skip_slow)
  

• Usage:

  • Run skipping slow tests (default):

    $ pytest -rs
    

  • Run including slow tests:

    $ pytest --runslow
    

5. Writing Well Integrated Assertion Helpers

You can write helper functions that fail tests with clear messages and hide themselves from tracebacks unless `--full-trace` is used.

• Example helper using __tracebackhide__:

  import pytest
  
  def checkconfig(x):
      __tracebackhide__ = True
      if not hasattr(x, "config"):
          pytest.fail(f"not configured: {x}")
  
  def test_something():
      checkconfig(42)
  

• Advanced usage:

  __tracebackhide__ can be a callable that filters exceptions, e.g., only hide tracebacks for certain exceptions.

6. Detecting if Running Inside a Pytest Run

Application code can detect if it is running under pytest by checking the environment variable `PYTEST_VERSION`.

import os

if os.environ.get("PYTEST_VERSION") is not None:
    # Running under pytest
    ...
else:
    # Normal execution
    ...

7. Adding Information to Test Report Header

Customize pytest's test session header by implementing the `pytest_report_header` hook.

• Simple example:

  def pytest_report_header(config):
      return "project deps: mylib-1.1"
  

• Conditional multi-line example:

  def pytest_report_header(config):
      if config.get_verbosity() > 0:
          return ["info1: did you know that ...", "did you?"]
  

8. Profiling Test Duration

Use `--durations=N` option to report the slowest N tests.

• Example tests:

  import time
  
  def test_funcfast():
      time.sleep(0.1)
  
  def test_funcslow1():
      time.sleep(0.2)
  
  def test_funcslow2():
      time.sleep(0.3)
  

• Run:

  $ pytest --durations=3
  

9. Incremental Testing / Test Steps

Implement an `incremental` marker on test classes to skip tests after a failure in the same class.

• conftest.py:

  from typing import Dict, Tuple
  import pytest
  
  _test_failed_incremental: Dict[str, Dict[Tuple[int, ...], str]] = {}
  
  def pytest_runtest_makereport(item, call):
      if "incremental" in item.keywords:
          if call.excinfo is not None:
              cls_name = str(item.cls)
              parametrize_index = tuple(item.callspec.indices.values()) if hasattr(item, "callspec") else ()
              test_name = item.originalname or item.name
              _test_failed_incremental.setdefault(cls_name, {}).setdefault(parametrize_index, test_name)
  
  def pytest_runtest_setup(item):
      if "incremental" in item.keywords:
          cls_name = str(item.cls)
          if cls_name in _test_failed_incremental:
              parametrize_index = tuple(item.callspec.indices.values()) if hasattr(item, "callspec") else ()
              test_name = _test_failed_incremental[cls_name].get(parametrize_index, None)
              if test_name is not None:
                  pytest.xfail(f"previous test failed ({test_name})")
  

• Usage in test class:

  import pytest
  
  @pytest.mark.incremental
  class TestUserHandling:
      def test_login(self):
          pass
  
      def test_modification(self):
          assert 0  # fails
  
      def test_deletion(self):
          pass  # skipped due to previous failure
  

10. Package/Directory-level Fixtures (Setups)

Define fixtures scoped to a directory or package by placing them in `conftest.py` inside that directory.

• Example:

  # a/conftest.py
  import pytest
  
  class DB:
      pass
  
  @pytest.fixture(scope="package")
  def db():
      return DB()
  

• Tests in the same directory can use db fixture; tests in sibling directories cannot unless a similar fixture is defined there.

11. Post-process Test Reports / Failures

You can hook into `pytest_runtest_makereport` to post-process test results, e.g., logging all failures to a file.

• Example conftest.py:

  import os.path
  import pytest
  
  @pytest.hookimpl(wrapper=True, tryfirst=True)
  def pytest_runtest_makereport(item, call):
      rep = yield
      if rep.when == "call" and rep.failed:
          mode = "a" if os.path.exists("failures") else "w"
          with open("failures", mode, encoding="utf-8") as f:
              extra = f" ({item.funcargs['tmp_path']})" if "tmp_path" in item.fixturenames else ""
              f.write(rep.nodeid + extra + "\n")
      return rep
  

12. Making Test Result Information Available in Fixtures

Fixtures can access detailed test results after test execution using stash keys and the `pytest_runtest_makereport` hook.

• Example conftest.py:

  from typing import Dict
  import pytest
  from pytest import StashKey, CollectReport
  
  phase_report_key = StashKey[Dict[str, CollectReport]]()
  
  @pytest.hookimpl(wrapper=True, tryfirst=True)
  def pytest_runtest_makereport(item, call):
      rep = yield
      item.stash.setdefault(phase_report_key, {})[rep.when] = rep
      return rep
  
  @pytest.fixture
  def something(request):
      yield
      report = request.node.stash[phase_report_key]
      if report["setup"].failed:
          print("setting up a test failed", request.node.nodeid)
      elif report["setup"].skipped:
          print("setting up a test skipped", request.node.nodeid)
      elif ("call" not in report) or report["call"].failed:
          print("executing test failed or skipped", request.node.nodeid)
  

13. PYTEST_CURRENT_TEST Environment Variable

During test execution, pytest sets the environment variable `PYTEST_CURRENT_TEST` containing the current test node id and phase (`setup`, `call`, or `teardown`). This aids in diagnosing stuck tests via external monitoring tools.

Example usage with `psutil`:

import psutil

for pid in psutil.pids():
    environ = psutil.Process(pid).environ()
    if "PYTEST_CURRENT_TEST" in environ:
        print(f'pytest process {pid} running: {environ["PYTEST_CURRENT_TEST"]}')

14. Freezing Pytest (Packaging)

When freezing applications with tools like PyInstaller, you can package pytest inside your executable to allow running tests on end-user machines.

• You can embed pytest and plugins explicitly since plugin discovery via entry points won't work in frozen executables.

• Example entry point script snippet:

  import sys
  import pytest_timeout  # example third party plugin
  
  if len(sys.argv) > 1 and sys.argv[1] == "--pytest":
      import pytest
      sys.exit(pytest.main(sys.argv[2:], plugins=[pytest_timeout]))
  else:
      # normal app code
      ...
  

Usage:

./app_main --pytest --verbose --tb=long --junit=xml=results.xml test-suite/

Important Implementation Details

• Many examples rely on pytest's hook system (e.g., pytest_addoption, pytest_collection_modifyitems, pytest_runtest_makereport) to customize behavior.

• Fixtures are used extensively to inject command line options or share state.

• The incremental testing pattern uses a dictionary keyed by class name and parametrization index to track failures and skip subsequent tests.

• Traceback hiding via __tracebackhide__ allows cleaner failure output.

• Custom CLI options can be validated with choices or type parameters raising pytest.UsageError.

• Post-processing test reports enables custom logging or failure handling.

• Environment variables set by pytest (PYTEST_VERSION, PYTEST_CURRENT_TEST) provide runtime context.

• Packaging pytest inside frozen executables requires explicit plugin imports.

Interactions with Other Parts of the System

• This file is tightly coupled with pytest internals and plugin/hook APIs.

• It shows how to write conftest.py files that extend pytest behavior per project or directory.

• It demonstrates how tests themselves can consume custom fixtures and options added via conftest.py.

• The examples integrate with external tools/libraries like psutil for process inspection.

• The freezing example interacts with packaging tools like PyInstaller, enabling pytest to run inside frozen apps.

• Custom markers (slow, incremental) and fixtures can be used by tests to control execution flow.

• The file references pytest command line options and environment variables to influence test runs.

Visual Diagram (Class Diagram of Key Hook Implementations and Fixtures)

classDiagram
    class Conftest {
        +pytest_addoption(parser)
        +pytest_configure(config)
        +pytest_collection_modifyitems(config, items)
        +pytest_runtest_makereport(item, call)
        +pytest_runtest_setup(item)
        +pytest_report_header(config)
        +cmdopt(request)
        +db()
        +something(request)
    }

    class IncrementalState {
        -_test_failed_incremental: Dict[str, Dict[Tuple[int, ...], str]]
    }
    Conftest ..> IncrementalState : uses

    class DB {
    }

    Conftest ..> DB : fixture provides

    class pytest {
        <<framework>>
        +fail(message)
        +mark.skip(reason)
        +xfail(reason)
    }

    Conftest ..> pytest : uses hooks and markers

    class Request {
        +config
        +getoption(name)
        +node
        +funcargs
        +stash
    }

    Conftest ..> Request : fixture parameter

This diagram shows the main entities and their relationships in the examples: the `conftest.py` file implements multiple pytest hook functions and fixtures, uses internal state (`_test_failed_incremental`) for incremental tests, and interacts with pytest's core API.

Summary

The [simple.rst](/projects/286/67458) file is an extensive tutorial and pattern collection for customizing and extending pytest. It covers practical techniques for:

• Custom CLI options and fixtures

• Dynamic test skipping and selection

• Test profiling and incremental test management

• Post-processing test results and reporting

• Integrating pytest with packaging and frozen executables

• Using environment variables and internal pytest features for diagnostics

The file uses a mixture of concise code examples, shell command snippets, and detailed explanations, making it an invaluable resource for pytest users aiming to implement advanced test workflows and integrations.

This documentation should enable users to understand and apply the demonstrated pytest customization patterns to their own testing projects effectively.