usage.rst

Overview

This file serves as a comprehensive usage guide for **pytest**, a popular Python testing framework. It covers various methods of invoking pytest from the command line and from Python code, explains how to select and run specific tests, and provides options for managing plugins and profiling test execution durations.

The document is structured as a reStructuredText (reST) file intended for integration into pytest’s official documentation. It is primarily focused on users who want to understand how to execute tests effectively and customize pytest behavior via command-line options or programmatic interfaces.

Detailed Explanations

How to invoke pytest

• The simplest way to run tests is by executing the command pytest in the terminal.

• By default, pytest discovers and runs all test files matching test_*.py or *_test.py patterns in the current directory and its subdirectories.

• The test discovery follows pytest’s standard test discovery rules.

Specifying which tests to run

Pytest supports flexible test selection approaches, allowing users to run tests selectively by module, directory, keyword expressions, markers, or parameterization.

1. Run tests in a module

pytest test_mod.py

Runs all tests in the specified module `test_mod.py`.

2. Run tests in a directory

pytest testing/

Runs tests recursively in the `testing/` directory.

3. Run tests by keyword expressions

pytest -k 'MyClass and not method'

• Runs tests whose names match the keyword expression (case-insensitive).

• Supports Python logical operators (and, or, not).

• Example: Runs TestMyClass.test_something but excludes TestMyClass.test_method_simple.

• On Windows, use double quotes "..." instead of single quotes '...'.

4. Run tests by collection arguments (Node IDs)

Node IDs specify the path to tests with optional class, method, and parameter selections.

• Run a specific test function:

  pytest tests/test_mod.py::test_func
  

• Run all tests in a class:

  pytest tests/test_mod.py::TestClass
  

• Run a specific test method in a class:

  pytest tests/test_mod.py::TestClass::test_method
  

• Run a specific parameterization of a test:

  pytest tests/test_mod.py::test_func[x1,y2]
  

5. Run tests by marker expressions

Markers are decorators that tag tests for selective running.

• Run tests marked with @pytest.mark.slow:

  pytest -m slow
  

• Run tests with a marker that has keyword arguments, e.g. @pytest.mark.slow(phase=1):

  pytest -m "slow(phase=1)"
  

Refer to the [marks documentation](#mark) for more details.

6. Run tests from packages

pytest --pyargs pkg.testing

• Imports pkg.testing and runs tests found in its filesystem location.

7. Read arguments from file (added in version 8.2)

You can store test selection arguments in a file and use the `@` prefix to read them:

pytest @tests_to_run.txt

Contents of `tests_to_run.txt` example:

tests/test_file.py
tests/test_mod.py::test_func[x1,y2]
tests/test_mod.py::TestClass
-m slow

The file can be generated via:

pytest --collect-only -q

and modified as needed.

Getting help on version, options, and environment variables

pytest --version    # Shows pytest version and import location
pytest --fixtures   # Shows available builtin fixtures (function arguments)
pytest -h | --help  # Displays help for command line and config file options

Profiling test execution duration

(Pytest 6.0 behavior changed here)

To list the slowest 10 tests taking longer than 1.0 seconds:

pytest --durations=10 --durations-min=1.0

By default, pytest hides very fast test durations (<0.005s) unless the verbose flag `-vv` is given.

Managing loading of plugins

Early loading plugins

Use the `-p` option to load plugins explicitly at startup:

pytest -p mypluginmodule

The `name` can be:

• A fully qualified module dotted path (e.g., myproject.plugins).

• The entry-point plugin name (e.g., pytest_cov for the pytest-cov plugin):

pytest -p pytest_cov

Disabling plugins

Disable a plugin by prefixing its name with `no:` in the `-p` option:

pytest -p no:doctest

This disables the `doctest` plugin responsible for running doctests.

Other ways of calling pytest

Calling pytest through python -m pytest

python -m pytest [...]

• Equivalent to running pytest [...] directly.

• Adds the current directory to sys.path.

Calling pytest from Python code

import pytest

retcode = pytest.main()

• Runs pytest programmatically, returning an exit code instead of raising SystemExit.

• By default, reads arguments from sys.argv.

• You can pass a list of arguments explicitly:

retcode = pytest.main(["-x", "mytestdir"])

• You can also specify additional plugins by providing them as instances:

class MyPlugin:
    def pytest_sessionfinish(self):
        print("*** test run reporting finishing")

retcode = pytest.main(["-qq"], plugins=[MyPlugin()])

• Result: The plugin hook is invoked during the test session finish.

**Note:** Calling `pytest.main()` multiple times in the same process is discouraged because Python’s import caching means changes to tests or modules are not reflected on subsequent calls.

Implementation Details and Algorithms

This file is a static documentation file written in reStructuredText format. It does not contain executable code, classes, or functions. Instead, it provides usage instructions, command-line examples, and explanations for invoking and configuring pytest.

The documentation uses Sphinx directives to structure the content with sections, cross-references, code blocks (bash, python, text), and notes.

The key "algorithmic" content is embedded in the explanations of pytest’s test discovery and selection mechanisms, including:

• Test discovery patterns (test_*.py and *_test.py)

• Keyword expression parsing (-k)

• Node ID syntax for selecting tests (:: separator and parameterization)

• Marker-based test selection (-m)

• Reading options from files with @ prefix

These mechanisms are part of pytest itself (outside this file) but are described here for user reference.

Interactions with Other Parts of the System

• This file is part of the official pytest documentation set and interacts by providing user-facing instructions linked from other docs such as:

  • command-line-flags

  • invoke-other

  • test discovery

  • marks

• It references pytest’s command line interface and Python API (pytest.main()).

• It explains how to invoke pytest plugins and manage them via command-line options, which ties into pytest’s plugin architecture.

• It describes how to generate and use argument files that can be created with pytest’s --collect-only option.

• The examples and explanations here assume the presence of test files following pytest conventions in the user’s project directories.

Visual Diagram

Below is a **flowchart** representing the main usage workflows and options described in this file.

flowchart TD
    A[Invoke pytest] --> B[Default: pytest]
    A --> C[Run tests selectively]
    A --> D[Manage plugins]
    A --> E[Get help & info]
    A --> F[Invoke from Python code]

    C --> C1[Run tests in module]
    C --> C2[Run tests in directory]
    C --> C3[Run tests by keyword (-k)]
    C --> C4[Run tests by node ID (::)]
    C --> C5[Run tests by marker (-m)]
    C --> C6[Run tests from packages (--pyargs)]
    C --> C7[Read args from file (@file)]

    D --> D1[Early-load plugins (-p name)]
    D --> D2[Disable plugins (-p no:name)]

    E --> E1[--version]
    E --> E2[--fixtures]
    E --> E3[-h / --help]

    F --> F1[pytest.main()]
    F --> F2[pytest.main(args)]
    F --> F3[pytest.main(args, plugins=[...])]

    style A fill:#f9f,stroke:#333,stroke-width:2px
    style C fill:#bbf,stroke:#333,stroke-width:1px
    style D fill:#bfb,stroke:#333,stroke-width:1px
    style E fill:#fbf,stroke:#333,stroke-width:1px
    style F fill:#ffb,stroke:#333,stroke-width:1px

Usage Examples

# Run all tests in the current directory and subdirectories
pytest

# Run tests in a specific file
pytest tests/test_example.py

# Run tests in a directory
pytest tests/

# Run tests matching keyword expression
pytest -k "MyClass and not method"

# Run a specific test function in a module
pytest tests/test_example.py::test_func

# Run all tests in a class
pytest tests/test_example.py::TestClass

# Run tests marked as slow
pytest -m slow

# Run tests from a package
pytest --pyargs mypackage.tests

# Read tests to run from a file
pytest @mytests.txt

# Run pytest programmatically with a plugin
import pytest

class CustomPlugin:
    def pytest_sessionfinish(self):
        print("Finished")

pytest.main(["-v"], plugins=[CustomPlugin()])

This documentation aims to provide users with clear, practical instructions for executing pytest and customizing test runs, covering both command-line and programmatic use cases.