cacheprovider.py

Overview

The `cacheprovider.py` file implements the core caching mechanism and related plugins for **pytest** that enable persistent storage of test session data between runs. This caching system primarily supports features such as:

The cache stores JSON-serializable data about test outcomes and other plugin state in a dedicated directory (default `.pytest_cache`). This enables pytest to optimize test runs, focusing on potential problem areas, reducing feedback loops, and improving developer productivity.

The file also defines pytest CLI options, fixtures, and plugin hook implementations to integrate caching functionality seamlessly into the pytest lifecycle.

Classes, Functions, and Methods

Cache class

A persistent cache interface exposed via the `cache` fixture, allowing plugins and tests to store and retrieve JSON-serializable data across test sessions.

Purpose

Manages a structured cache directory with separate subdirectories for:

Declaration Summary

@dataclasses.dataclass
class Cache:
    _cachedir: Path
    _config: Config

__init__(self, cachedir: Path, config: Config, *, _ispytest: bool = False)

Initializes the cache instance with the cache directory and pytest `Config` object.

@classmethod for_config(cls, config: Config, *, _ispytest: bool = False) -> Cache

Creates a `Cache` instance for a given pytest `Config`. Clears the cache directory if the `--cache-clear` option is set.

cache = Cache.for_config(config)

@classmethod clear_cache(cls, cachedir: Path, _ispytest: bool = False) -> None

Clears cache subdirectories that hold cached directories and values.

@staticmethod cache_dir_from_config(config: Config, *, _ispytest: bool = False) -> Path

Resolves the cache directory path from the pytest configuration.

warn(self, fmt: str, *, _ispytest: bool = False, **args: object) -> None

Issues a pytest cache-related warning via pytest's warning system.

_mkdir(self, path: Path) -> None

Internal helper to create directories under the cache directory, ensuring supporting files exist.

mkdir(self, name: str) -> Path

Creates (if necessary) and returns a directory path under the cache for a given name.

cache_dir = cache.mkdir("myplugin")
# Use cache_dir to store plugin files

_getvaluepath(self, key: str) -> Path

Returns the path to the cached value file for a given cache key.

get(self, key: str, default)

Retrieves a cached value by key. Returns `default` if the key is missing or the stored data is invalid.

lastfailed = cache.get("cache/lastfailed", default={})

set(self, key: str, value: object) -> None

Stores a JSON-serializable `value` under the given key.

cache.set("cache/lastfailed", lastfailed_dict)

_ensure_cache_dir_and_supporting_files(self) -> None

Creates the cache directory if it does not exist, along with supporting files:

This ensures proper cache directory structure and Git compatibility.

LFPlugin

**Purpose:** Implements the `--lf` (last-failed) and `--ff` (failed-first) pytest options.

Key Attributes:

Important Methods:

LFPluginCollWrapper

A wrapper plugin for LFPlugin that modifies test collection reports to prioritize or filter tests for last-failed behavior.

LFPluginCollSkipfiles

Plugin that skips files during collection if they do not contain previously failed tests, supporting `--lf` efficiency.

NFPlugin

**Purpose:** Implements the `--nf` (new-first) pytest option.

Key Attributes:

Important Methods:

pytest CLI Option Registration

Function: `pytest_addoption(parser: Parser)`

Defines command-line options and an ini option related to caching:

pytest Hooks and Fixtures

cacheshow(config, session)

Implements the `--cache-show` command to list cached values and directories matching an optional glob pattern.

Important Implementation Details and Algorithms

Interaction with Other Parts of the System

Usage Examples

Using the Cache Fixture in a Plugin or Test

def test_something(cache):
    data = cache.get("myplugin/data", default={})
    # ... modify data ...
    cache.set("myplugin/data", data)

Running Only Last Failed Tests

pytest --lf

This command will run only tests that failed in the previous pytest run.

Running Failed Tests First, Then All Others

pytest --ff

This runs all tests, but schedules previously failed tests to run before others.

Mermaid Diagram: Class Structure of cacheprovider.py

classDiagram
    class Cache {
        -_cachedir: Path
        -_config: Config
        -_CACHE_PREFIX_DIRS: str
        -_CACHE_PREFIX_VALUES: str
        +__init__(cachedir: Path, config: Config, _ispytest: bool)
        +for_config(config: Config, _ispytest: bool) Cache
        +clear_cache(cachedir: Path, _ispytest: bool) void
        +cache_dir_from_config(config: Config, _ispytest: bool) Path
        +warn(fmt: str, _ispytest: bool, **args) void
        +mkdir(name: str) Path
        +get(key: str, default) object
        +set(key: str, value: object) void
        -_mkdir(path: Path) void
        -_getvaluepath(key: str) Path
        -_ensure_cache_dir_and_supporting_files() void
    }

    class LFPlugin {
        -config: Config
        -active: bool
        -lastfailed: dict
        -_last_failed_paths: set
        -_skipped_files: int
        +__init__(config: Config) void
        +get_last_failed_paths() set
        +pytest_collection_modifyitems(config: Config, items: list) Generator
        +pytest_runtest_logreport(report: TestReport) void
        +pytest_collectreport(report: CollectReport) void
        +pytest_sessionfinish(session: Session) void
        +pytest_report_collectionfinish() str | None
    }

    class LFPluginCollWrapper {
        -lfplugin: LFPlugin
        -_collected_at_least_one_failure: bool
        +__init__(lfplugin: LFPlugin) void
        +pytest_make_collect_report(collector: Collector) Generator
    }

    class LFPluginCollSkipfiles {
        -lfplugin: LFPlugin
        +__init__(lfplugin: LFPlugin) void
        +pytest_make_collect_report(collector: Collector) CollectReport | None
    }

    class NFPlugin {
        -config: Config
        -active: bool
        -cached_nodeids: set
        +__init__(config: Config) void
        +pytest_collection_modifyitems(items: list) Generator
        +pytest_sessionfinish() void
        -_get_increasing_order(items: Iterable) list
    }

    Cache <-- LFPlugin : uses
    LFPlugin <.. LFPluginCollWrapper : wraps
    LFPlugin <.. LFPluginCollSkipfiles : supports
    Cache <-- NFPlugin : uses

Summary

The `cacheprovider.py` file is a foundational component in pytest that provides persistent caching services and plugins to optimize test execution based on historical test results. It enables selective reruns of failed tests, prioritization of test execution order, and offers a stable, JSON-backed cache interface accessible to other plugins and tests.

This system improves test efficiency, reduces wait times for developers, and integrates tightly with pytest’s plugin and hook infrastructure.

If you need further details or examples for specific classes or functions, please feel free to ask!