parametrize.rst

Overview

This documentation file explains how to use **parametrization** features in `pytest`, a popular Python testing framework. Parametrization allows running a single test function or fixture multiple times with different argument values, enabling more comprehensive and maintainable test suites.

The file covers:

• Basic usage of @pytest.mark.parametrize decorator to define multiple test inputs.

• Parametrizing test classes and entire test modules.

• Using pytest.param to mark individual test cases with special behavior (e.g., expected failure).

• Stacking multiple parametrize decorators to produce combinatorial argument sets.

• Advanced dynamic parametrization with the pytest_generate_tests hook.

• Notes on parameter value handling and Unicode escaping.

• Examples illustrating the concepts and typical workflows.

Detailed Explanation

@pytest.mark.parametrize decorator

The primary mechanism for test function parametrization is the `@pytest.mark.parametrize` decorator. It allows specifying multiple sets of arguments that a test function should be executed with.

Syntax

@pytest.mark.parametrize(argnames, argvalues)
def test_func(arg1, arg2, ...):
    ...

• argnames: A string of comma-separated argument names (e.g., "test_input,expected") or a list/tuple of argument names.

• argvalues: A list of tuples or values corresponding to each argument set.

Example

import pytest

@pytest.mark.parametrize("test_input,expected", [("3+5", 8), ("2+4", 6), ("6*9", 42)])
def test_eval(test_input, expected):
    assert eval(test_input) == expected

This runs `test_eval` three times with each pair of inputs.

Important Notes

• Parameter values are passed as-is; mutations in one test call affect subsequent calls if mutable objects are used.

• By default, pytest escapes non-ASCII characters in parameter string representations for terminal output. This can be disabled at your own risk via pytest.ini:

[pytest]
disable_test_id_escaping_and_forfeit_all_rights_to_community_support = True

Parametrizing Test Classes and Modules

You can apply `@pytest.mark.parametrize` to a test class to run all its test methods multiple times with different argument sets:

import pytest

@pytest.mark.parametrize("n,expected", [(1, 2), (3, 4)])
class TestClass:
    def test_simple_case(self, n, expected):
        assert n + 1 == expected

    def test_weird_simple_case(self, n, expected):
        assert (n * 1) + 1 == expected

Similarly, to parametrize all tests in a module, assign the decorator to the global `pytestmark` variable:

import pytest

pytestmark = pytest.mark.parametrize("n,expected", [(1, 2), (3, 4)])

class TestClass:
    def test_simple_case(self, n, expected):
        assert n + 1 == expected

    def test_weird_simple_case(self, n, expected):
        assert (n * 1) + 1 == expected

Marking Individual Parameter Sets

You can mark specific parameter values with special markers such as `xfail` to indicate expected failures:

import pytest

@pytest.mark.parametrize(
    "test_input,expected",
    [
        ("3+5", 8),
        ("2+4", 6),
        pytest.param("6*9", 42, marks=pytest.mark.xfail),
    ],
)
def test_eval(test_input, expected):
    assert eval(test_input) == expected

Stacking Parametrize Decorators

Stacking multiple `@pytest.mark.parametrize` decorators produces the Cartesian product of all argument sets:

import pytest

@pytest.mark.parametrize("x", [0, 1])
@pytest.mark.parametrize("y", [2, 3])
def test_foo(x, y):
    pass

This runs the test with argument combinations:

• x=0, y=2

• x=1, y=2

• x=0, y=3

• x=1, y=3

pytest_generate_tests Hook for Dynamic Parametrization

For more advanced or dynamic parametrization, `pytest` provides the `pytest_generate_tests` hook. This function is called during test collection and allows programmatically specifying parameter sets.

Example

Suppose you want to parametrize a test based on command-line options:

• Test function using a fixture stringinput:

def test_valid_string(stringinput):
    assert stringinput.isalpha()

• conftest.py to add command-line option and parametrization hook:

def pytest_addoption(parser):
    parser.addoption(
        "--stringinput",
        action="append",
        default=[],
        help="list of stringinputs to pass to test functions",
    )

def pytest_generate_tests(metafunc):
    if "stringinput" in metafunc.fixturenames:
        metafunc.parametrize("stringinput", metafunc.config.getoption("stringinput"))

Run tests with:

pytest -q --stringinput="hello" --stringinput="world" test_strings.py

Implementation Details and Algorithms

• The @pytest.mark.parametrize decorator internally registers multiple argument sets with pytest’s test collection mechanism, causing multiple test calls with different parameters.

• Stacking decorators leads to the Cartesian product of parameters, enabling combinatorial testing.

• The pytest_generate_tests hook provides a flexible way to inject parameters dynamically based on runtime information such as command-line options or external data sources.

• Parameter values are passed by reference, so mutable objects can be shared across test invocations, which requires care to avoid side effects.

• Unicode escaping in parameter identifiers ensures compatibility across different terminals and OS environments but can be optionally disabled.

Interactions with Other System Components

• This file documents features of the pytest testing framework, which integrates with Python codebases to run unit and functional tests.

• Parametrization interacts closely with:

  • pytest.fixture for parameterizing fixtures.

  • pytest.mark for test markers (e.g., xfail, skip).

  • pytest.ini configuration for controlling behaviors like Unicode escaping.

  • Test discovery and collection system that expands parameter sets into individual test cases.

• It supports extension points such as pytest_generate_tests for custom test generation strategies.

• Parametrized tests produce multiple test items reported individually by pytest’s test runner and UI.

Usage Examples Summary

1. Basic parametrized test function:

   @pytest.mark.parametrize("input,expected", [("1+1", 2), ("2*2", 4)])
   def test_math(input, expected):
       assert eval(input) == expected
   

2. Parametrized test class:

   @pytest.mark.parametrize("n,expected", [(1, 2), (3, 4)])
   class TestExample:
       def test_increment(self, n, expected):
           assert n + 1 == expected
   

3. Mark individual parameter with expected failure:

   @pytest.mark.parametrize(
       "expr,result",
       [("2+2", 4), pytest.param("3+3", 7, marks=pytest.mark.xfail)]
   )
   def test_expr(expr, result):
       assert eval(expr) == result
   

4. Dynamic parametrization with command line:

   # conftest.py
   def pytest_addoption(parser):
       parser.addoption("--data", action="append", default=[])
   
   def pytest_generate_tests(metafunc):
       if "data" in metafunc.fixturenames:
           metafunc.parametrize("data", metafunc.config.getoption("data"))
   

Mermaid Diagram: Parametrization Workflow Flowchart

flowchart TD
    A[Test Function] -->|Decorated with| B[@pytest.mark.parametrize]
    B --> C{Multiple Parameter Sets}
    C --> D[Test Runs with Param Set 1]
    C --> E[Test Runs with Param Set 2]
    C --> F[Test Runs with Param Set N]

    G[pytest_generate_tests Hook] --> H[Inspect metafunc]
    H --> I{Parametrize dynamically?}
    I -->|Yes| J[metafunc.parametrize(...) called]
    J --> K[Additional Test Runs]

    style A fill:#f9f,stroke:#333,stroke-width:2px
    style B fill:#bbf,stroke:#333,stroke-width:2px
    style G fill:#bbf,stroke:#333,stroke-width:2px

• The diagram shows how the @pytest.mark.parametrize decorator expands test functions into multiple test runs with different parameter sets.

• It also illustrates the dynamic parametrization flow via the pytest_generate_tests hook, enabling additional parameter sets during test collection.

References

• pytest official docs on parametrization: https://docs.pytest.org/en/stable/how-to/parametrize.html

• pytest fixtures and markers for advanced test setups.

• pytest command-line options and configuration files (pytest.ini).

This document serves as a comprehensive guide to understand and effectively use parametrization in pytest to write concise, powerful, and flexible test suites.