structures.py

Overview

The [structures.py](/projects/286/67331) file is a core utility module within the pytest testing framework that defines fundamental data structures and mechanisms for managing pytest marks and parameter sets. Marks are metadata annotations applied to test functions or classes to influence test discovery, execution, and reporting (e.g., skip, xfail, parametrize). Parameter sets represent collections of input values used in parameterized tests.

This file provides:

• The ParameterSet class to encapsulate parameter values, associated marks, and optional IDs.

• The Mark and MarkDecorator classes to represent and apply pytest marks.

• The MarkGenerator singleton for creating mark decorators dynamically (exposed as pytest.mark).

• Utilities for storing, retrieving, and normalizing marks on test objects.

• The NodeKeywords mapping to manage keywords (markers) associated with test nodes, supporting inheritance through the node hierarchy.

Together, these data structures and utilities underpin pytest's powerful marking and parameterization capabilities.

Detailed Explanations

Classes

ParameterSet (NamedTuple)

A `ParameterSet` represents a single set of parameter values for a parameterized test, along with any associated marks and an optional ID.

Attributes

• values: Sequence[object | NotSetType]
  The parameter values for this set.

• marks: Collection[MarkDecorator | Mark]
  Marks associated with this parameter set.

• id: str | _HiddenParam | None
  Optional identifier for the parameter set. Can be a string or the special singleton HIDDEN_PARAM used to hide the parameter set from test names.

Class Methods

• param(*values, marks=(), id=None) -> ParameterSet
  Factory method to create a ParameterSet instance. Validates input marks and ID.

  Parameters:

  • *values: Parameter values.

  • marks: A single MarkDecorator or a collection of marks to associate.

  • id: Optional string or HIDDEN_PARAM sentinel.

  **Returns:** A new `ParameterSet` instance.

  **Example:**

  pytest.param(1, 2, 3, marks=pytest.mark.xfail, id="mycase")
  

• extract_from(parameterset, force_tuple=False) -> ParameterSet
  Converts legacy or various input formats into a ParameterSet instance.

  Parameters:

  • parameterset: Can be an existing ParameterSet, sequence, or single object.

  • force_tuple: If True, wraps single values in a tuple to avoid decomposition.

  **Returns:** A `ParameterSet` instance.

• _parse_parametrize_args(argnames, argvalues, *args, **kwargs) -> tuple[Sequence[str], bool]
  Parses parametrize arguments, normalizing parameter names into a sequence and determining if tuple-wrapping is needed.

• _parse_parametrize_parameters(argvalues, force_tuple) -> list[ParameterSet]
  Converts iterable of parameter values into a list of normalized ParameterSet instances.

• _for_parametrize(argnames, argvalues, func, config, nodeid) -> tuple[Sequence[str], list[ParameterSet]]
  Main entry point for processing parameterization arguments, validating counts, and handling empty parameter sets by applying appropriate marks.

_HiddenParam (enum.Enum)

A private enum singleton used to denote hidden parameter sets (`HIDDEN_PARAM`). This allows a parameter set to be excluded from test names.

Mark (frozen dataclass)

Represents a pytest mark, which is metadata attached to tests.

Attributes

• name: str — Name of the mark (e.g., "skip", "xfail").

• args: tuple[Any, ...] — Positional arguments given to the mark decorator.

• kwargs: Mapping[str, Any] — Keyword arguments given to the mark decorator.

• _param_ids_from: Mark | None — Internal reference to mark source for parametrize IDs.

• _param_ids_generated: Sequence[str] | None — Internal cache of generated IDs.

Methods

• __init__(name, args, kwargs, param_ids_from=None, param_ids_generated=None, *, _ispytest=False)
  Internal constructor ensuring marks are created only by pytest.

• _has_param_ids() -> bool
  Returns True if the mark contains parameter IDs.

• combined_with(other: Mark) -> Mark
  Combines two marks of the same name by concatenating arguments and merging keyword arguments.

MarkDecorator (dataclass)

A decorator object used to apply `Mark` instances to test functions or classes.

Attributes

• mark: Mark — The underlying mark data.

Properties

• name — Alias for mark.name.

• args — Alias for mark.args.

• kwargs — Alias for mark.kwargs.

• markname — Backwards compatibility alias for name.

Methods

• __init__(mark: Mark, *, _ispytest: bool = False)
  Internal constructor.

• with_args(*args, **kwargs) -> MarkDecorator
  Returns a new MarkDecorator with additional arguments merged.

• __call__(*args, **kwargs)
  When called, applies the mark to a test function or class if called with a single callable/class argument and no kwargs. Otherwise returns a new MarkDecorator with added arguments.

  Usage Example:

  mark = pytest.mark.slow
  @mark
  def test_func():
      pass
  
  # or with args
  mark = pytest.mark.skip(reason="not implemented")
  @mark
  def test_func():
      pass
  

MarkGenerator

A factory class that dynamically generates `MarkDecorator` instances for arbitrary mark names. This class is exposed as the singleton `MARK_GEN` and as `pytest.mark`.

• Accessing any attribute (e.g., pytest.mark.slow) returns a MarkDecorator with that name.

• Validates marker names against known markers from the pytest configuration, warning or failing on unknown marks.

• Provides special typing for builtin marks (skip, skipif, xfail, parametrize, etc.) for type checking.

NodeKeywords (MutableMapping[str, Any])

A dictionary-like container mapping marker names to values for a given test node, supporting inheritance from parent nodes.

Attributes

• _markers: dict[str, Any] — Markers stored on this node.

• node: Node — Associated test node.

• parent: Node | None — Parent node in the test tree.

Methods

• __getitem__(key: str)
  Retrieves marker value. Falls back to parent node if not found locally.

• __setitem__(key: str, value: Any)
  Sets marker value locally.

• __contains__(key: object) -> bool
  Checks if marker exists locally or in parent.

• update(other, **kwds)
  Updates local markers.

• __delitem__(key: str)
  Disabled; raises ValueError to prevent deletion.

• __iter__()
  Iterates over unique keys from local and parent markers.

• __len__()
  Returns total count of unique markers.

• __repr__()
  String representation for debugging.

Functions

istestfunc(func) -> bool

Determines if a callable is a test function (i.e., callable with a name not equal to `""`).

get_empty_parameterset_mark(config, argnames, func) -> MarkDecorator

Returns an appropriate mark to apply when a parameterized test has an empty parameter set, based on the pytest configuration option `"empty_parameter_set_mark"`.

• May return skip, xfail, or raise a collection error.

get_unpacked_marks(obj, *, consider_mro=True) -> list[Mark]

Extracts all marks applied to an object (function or class). If the object is a class and `consider_mro` is `True`, marks from all superclasses in MRO order are aggregated.

normalize_mark_list(mark_list) -> Iterable[Mark]

Normalizes an iterable of `Mark` or `MarkDecorator` instances into a sequence of `Mark` objects, extracting the underlying mark from decorators.

store_mark(obj, mark: Mark, *, stacklevel=2) -> None

Stores a `Mark` on an object by appending it to the `pytestmark` attribute (a list). Emits a warning if the object is a fixture.

Important Implementation Details and Algorithms

• Mark Storage:
  Marks are stored as a list in the pytestmark attribute of test functions or classes. The store_mark function manages appending marks while preserving existing marks.

• Parameter Sets:
  The ParameterSet class carefully handles wrapping and unwrapping of parameter values to support legacy and current styles, including the special HIDDEN_PARAM sentinel to hide parameters from test names.

• Mark Combination:
  Mark.combined_with allows merging two marks of the same name, which is essential when stacking multiple decorators of the same mark type.

• MarkGenerator Validation:
  The MarkGenerator dynamically validates mark names against the pytest configuration's known markers, issuing warnings or errors on unknown marks to catch typos early.

• Keyword Inheritance:
  NodeKeywords implements a dictionary that merges markers from the current node and its ancestors, enabling flexible marker inheritance in test collection hierarchies.

Interactions with Other System Components

• Integration with pytest config:
  The file interacts with the Config object to retrieve configuration options (e.g., known markers, empty parameter set handling).

• Nodes and Collection:
  Uses Node objects to relate marks and keywords to test nodes in the collection tree.

• Fixtures:
  Checks for fixture markers to warn users when attempting to apply marks improperly.

• Mark Decorators:
  Works closely with the rest of pytest to apply and interpret marks that influence test execution (skip, xfail, parametrize, etc.).

• Outcome and Failure Handling:
  Uses pytest's fail function to raise errors during parameter validation or unknown mark access.

Usage Examples

import pytest

# Creating a parameter set with marks and id
param = pytest.param(1, 2, marks=pytest.mark.xfail, id="xfail_case")

# Applying a mark as a decorator
@pytest.mark.skip(reason="not implemented")
def test_feature():
    pass

# Using parametrize with ParameterSet
@pytest.mark.parametrize(
    ("a", "b"),
    [
        (1, 2),
        pytest.param(3, 4, id="special_case", marks=pytest.mark.xfail),
    ]
)
def test_sum(a, b):
    assert a + b >= 3

Mermaid Class Diagram

classDiagram
    class ParameterSet {
        +values: Sequence[object | NotSetType]
        +marks: Collection[MarkDecorator | Mark]
        +id: str | _HiddenParam | None
        +param(*values, marks=(), id=None)
        +extract_from(parameterset, force_tuple=False)
        +_parse_parametrize_args(argnames, argvalues)
        +_parse_parametrize_parameters(argvalues, force_tuple)
        +_for_parametrize(argnames, argvalues, func, config, nodeid)
    }

    class _HiddenParam {
        <<enum>>
        +token
    }

    class Mark {
        +name: str
        +args: tuple[Any, ...]
        +kwargs: Mapping[str, Any]
        +_param_ids_from: Mark | None
        +_param_ids_generated: Sequence[str] | None
        +combined_with(other: Mark) Mark
        +_has_param_ids() bool
    }

    class MarkDecorator {
        +mark: Mark
        +name: str
        +args: tuple[Any, ...]
        +kwargs: Mapping[str, Any]
        +with_args(*args, **kwargs) MarkDecorator
        +__call__(*args, **kwargs)
    }

    class MarkGenerator {
        +__getattr__(name: str) MarkDecorator
    }

    class NodeKeywords {
        -_markers: dict[str, Any]
        -node: Node
        -parent: Node
        +__getitem__(key: str) Any
        +__setitem__(key: str, value: Any)
        +__contains__(key: object) bool
        +update(other, **kwds)
        +__delitem__(key: str)
        +__iter__()
        +__len__()
    }

    ParameterSet ..> _HiddenParam : uses
    MarkDecorator --> Mark : wraps
    MarkGenerator --> MarkDecorator : generates
    NodeKeywords ..> "Node" : aggregates

Summary

[structures.py](/projects/286/67331) is a foundational module within pytest that defines how marks and parameter sets are represented, stored, combined, and applied to tests. It provides flexible, extensible classes and methods that enable pytest's powerful marking and parameterization system, including dynamic mark creation, parameter set management, and marker inheritance. This file is integral to the test metadata and parameter processing pipeline in pytest.