markers.rst

Overview

This documentation file provides an extensive guide on using **pytest markers**—a powerful feature in the pytest testing framework that allows users to add metadata and selectively run tests based on custom or built-in markers.

Markers enable flexible test selection, grouping, and conditional execution by associating tests with labels or expressions. This file explains how to apply markers, register custom markers, use marker expressions for test selection, and advanced usage such as scoped marking, parameterized test marking, and plugin-defined markers.

It also covers practical examples, command line usages, and best practices for marker management in pytest test suites.

Detailed Descriptions

Key Concepts

Marker: A label or metadata attached to test functions, classes, or modules.
Node ID: A unique identifier for test items, typically in the form module.py::ClassName::test_method.
Marker Expression: A logical expression used to select tests based on markers and their arguments.
Keyword Expression: Filtering tests by substring matching of names using the -k option.
Parametrize: Running the same test function multiple times with different arguments.

Sections & Examples

Marking Test Functions and Selecting Tests

You can mark test functions by decorating them with `@pytest.mark.`. Markers may also accept keyword arguments.

import pytest

@pytest.mark.webtest
def test_send_http():
    pass

@pytest.mark.device(serial="123")
def test_something_quick():
    pass

**Selecting tests by marker:**

Run only tests marked webtest:
```
pytest -v -m webtest
```
Run all except webtest tests:
```
pytest -v -m "not webtest"
```
Run tests with marker device and keyword argument serial="123":
```
pytest -v -m "device(serial='123')"
```

**Notes:**

Only keyword argument matching is supported.
Supported types in marker expressions: int, str (unescaped), bool, and None.

Selecting Tests by Node ID

Node IDs allow precise selection of tests by module, class, or method names.

Examples:

Run a specific test method:

pytest -v test_server.py::TestClass::test_method

Run all tests in a class:
```
pytest -v test_server.py::TestClass
```

Run multiple specific tests:

pytest -v test_server.py::TestClass test_server.py::test_send_http

**Node ID format:**

module.py::function
module.py::Class::method
Parameterized tests include parameter values: module.py::function[param]

Selecting Tests by Name with `-k`

The `-k` option allows substring matching (case-insensitive) on test names, marker names, parent names, and extra keywords.

Examples:

Run tests with "http" in their names or markers:
```
pytest -v -k http
```
Run tests excluding those matching "send_http":
```
pytest -k "not send_http" -v
```
Run tests matching either "http" or "quick":
```
pytest -k "http or quick" -v
```

Logical operators supported: `and`, `or`, `not` with parentheses for grouping.

Registering Markers

To avoid typos and maintain clarity, markers should be registered in the pytest configuration file (`pytest.ini`):

[pytest]
markers =
    webtest: mark a test as a webtest.
    slow: mark test as slow.

You can list all registered markers with:

pytest --markers

Scoped Marking: Classes and Modules

Markers can be applied at class or module level to mark all contained tests.

Class-level:

@pytest.mark.webtest
class TestClass:
    def test_startup(self):
        pass

Module-level:

import pytest
pytestmark = pytest.mark.webtest

Multiple markers can be applied by setting `pytestmark` to a list.

Marking Individual Parameterized Tests

With `@pytest.mark.parametrize`, you can mark individual test cases differently:

@pytest.mark.foo
@pytest.mark.parametrize(
    ("n", "expected"), 
    [
        (1, 2), 
        pytest.param(1, 3, marks=pytest.mark.bar), 
        (2, 3)
    ]
)
def test_increment(n, expected):
    assert n + 1 == expected

Here, the `foo` marker applies to all cases; `bar` applies only to the second.

Custom Marker and Command Line Option Example

A plugin can add custom markers and command line options to control test runs:

Add option -E to run tests matching a named environment.
Add marker env(name) to mark tests for environments.

Example plugin in `conftest.py`:

import pytest

def pytest_addoption(parser):
    parser.addoption("-E", action="store", metavar="NAME",
                     help="only run tests matching the environment NAME.")

def pytest_configure(config):
    config.addinivalue_line(
        "markers", "env(name): mark test to run only on named environment"
    )

def pytest_runtest_setup(item):
    envnames = [mark.args[0] for mark in item.iter_markers(name="env")]
    if envnames and item.config.getoption("-E") not in envnames:
        pytest.skip(f"test requires env in {envnames!r}")

Usage:

pytest -E stage1

Passing Callables to Custom Markers

When passing a callable as a marker argument, use `.with_args` to avoid decorating the callable instead of passing it as an argument:

@pytest.mark.my_marker.with_args(hello_world)
def test_with_args():
    pass

Reading Multiple Marker Applications

Markers can be applied multiple times to the same test (e.g., on module, class, and function).

From plugin code, you can iterate all applied markers:

def pytest_runtest_setup(item):
    for mark in item.iter_markers(name="glob"):
        print(f"glob args={mark.args} kwargs={mark.kwargs}")

Platform-Specific Marking Example

A plugin can skip tests not matching the current platform marker:

import sys
import pytest

ALL = set("darwin linux win32".split())

def pytest_runtest_setup(item):
    supported_platforms = ALL.intersection(mark.name for mark in item.iter_markers())
    plat = sys.platform
    if supported_platforms and plat not in supported_platforms:
        pytest.skip(f"cannot run on platform {plat}")

Automatically Adding Markers Based on Test Names

Tests can be dynamically marked based on their names during collection:

def pytest_collection_modifyitems(items):
    for item in items:
        if "interface" in item.nodeid:
            item.add_marker(pytest.mark.interface)
        elif "event" in item.nodeid:
            item.add_marker(pytest.mark.event)

This allows use of `-m interface` or `-m event` to select subsets of tests.

Important Implementation Details

Markers are implemented as decorators attaching metadata to test functions or classes.
Test selection uses marker expressions parsed by pytest to filter tests during collection.
The pytestmark global variable allows module-level marker application.
The -k option uses substring expressions on test names and markers for filtering.
Markers can carry arguments (positional and keyword), but marker expressions only support keyword matching.
Custom markers should be registered in pytest.ini for discoverability and to avoid typos.
Plugins can extend marker functionality, add command line options, and dynamically add markers during test collection.
The pytest hook system (pytest_collection_modifyitems, pytest_runtest_setup) is used to implement dynamic behavior with markers.

Interactions with Other System Components

pytest CLI: Markers influence test selection and reporting based on command line options -m and -k.
pytest collection phase: Markers are evaluated and tests are filtered or skipped before execution.
pytest plugins: Can register custom markers and implement hook functions to modify test behavior depending on markers.
pytest configuration: Markers are registered via pytest.ini or programmatically in plugins.
Test modules and classes: Markers can be applied directly or inherited via class/module-level settings.

Usage Summary Examples

# Run only tests marked 'webtest'
pytest -v -m webtest

# Run tests that have marker 'device' with serial="123"
pytest -v -m "device(serial='123')"

# Run all tests except those containing 'send_http' in their name or markers
pytest -k "not send_http" -v

# Apply marker to all tests in a module
pytestmark = pytest.mark.slow

# Parametrized test with individual marks
@pytest.mark.foo
@pytest.mark.parametrize(
    ("n", "expected"),
    [(1, 2), pytest.param(1, 3, marks=pytest.mark.bar), (2, 3)]
)
def test_increment(n, expected):
    ...

Mermaid Diagram: Marker Usage Structure

The file is primarily documentation with no classes or functions defined. It revolves around pytest markers and their usage patterns.

The following flowchart illustrates the relationships between core concepts and usage workflows described:

flowchart TD
    A[Define Markers] --> B[Mark Test Functions / Classes / Modules]
    B --> C[Run Tests with Marker Selection (-m)]
    B --> D[Run Tests with Name Selection (-k)]
    B --> E[Parameterized Tests with Individual Marks]
    B --> F[Custom Markers via Plugins]
    F --> G[Add Command Line Options]
    F --> H[Add Marker Behavior in Hooks]
    C --> I[Test Collection Filters Tests by Marker Expression]
    D --> J[Substring Match on Test Names and Markers]
    I --> K[Selected Tests Run]
    J --> K
    H --> K

Summary

This file is a comprehensive guide on pytest markers, illustrating:

How to mark tests with built-in and custom markers.
Selecting tests by markers or test names.
Registering markers and best practices.
Advanced usage with parameterization, plugins, and dynamic marking.
Practical examples and CLI usage patterns.

It serves as an essential reference for pytest users to leverage markers for flexible, maintainable, and scalable test suites.

End of Documentation for markers.rst