test_cacheprovider.py

Overview

`test_cacheprovider.py` is a comprehensive test suite for verifying the caching functionality provided by pytest's cache provider system. This file contains multiple test classes and standalone test functions that ensure the correctness, robustness, and expected behavior of the cache API integrated within pytest's configuration and runtime environment.

The tests cover various scenarios including cache directory creation, permission handling, data serialization and retrieval, failure cases, integration with pytest's last-failed and new-first test ordering features, and the presence of auxiliary files like [.gitignore](/projects/286/67223) and [CACHEDIR.TAG](/projects/286/67341) in the cache directory.

The file relies heavily on pytest's testing utilities such as [Pytester](/projects/286/67470) for creating test environments and running pytest subprocesses, and [MonkeyPatch](/projects/286/67332) for environment manipulation. It interacts primarily with the [_pytest.cacheprovider.Cache](/projects/286/67341) class and pytest's configuration cache interface.

Classes and Their Responsibilities

1. TestNewAPI

Tests for the new cache API features and behaviors, including:

• Creation of cache directories using cache.mkdir().

• Permissions of the .pytest_cache directory.

• Error handling on invalid cache keys or data.

• Handling of cache write failures (e.g., due to unwritable directories).

• Integration of cache access in pytest configuration.

• Custom cache directory configuration via pytest.ini and environment variables.

• Functionality of the cache fixture in pytest tests.

**Key Methods:**

• test_config_cache_mkdir(pytester: Pytester) -> None
  Validates cache.mkdir() creates directories correctly and disallows keys with slashes.

• test_cache_dir_permissions(pytester: Pytester) -> None
  Ensures .pytest_cache directory permissions are as expected.

• test_config_cache_dataerror(pytester: Pytester) -> None
  Tests cache behavior when storing invalid data and retrieving with defaults.

• test_cache_writefail_cachefile_silent(pytester: Pytester) -> None
  Ensures cache failures due to corrupted files fail silently without crashing pytest.

• test_cache_writefail_permissions(unwritable_cache_dir: Path, pytester: Pytester) -> None
  Tests cache write failures due to directory permissions.

• test_cache_failure_warns(...) -> None
  Confirms that cache write failures emit a warning.

• test_config_cache(pytester: Pytester) -> None
  Checks that the config.cache attribute is available during test execution.

• test_cachefuncarg(pytester: Pytester) -> None
  Verifies usage of the cache test function argument for caching values.

• test_custom_rel_cache_dir(pytester: Pytester) -> None
  Tests custom relative cache directory configuration.

• test_custom_abs_cache_dir(pytester: Pytester, tmp_path_factory: TempPathFactory) -> None
  Tests custom absolute cache directory configuration.

• test_custom_cache_dir_with_env_var(pytester: Pytester, monkeypatch: MonkeyPatch) -> None
  Tests cache directory configuration with environment variable interpolation.

• unwritable_cache_dir(pytester: Pytester) -> Generator[Path]
  A pytest fixture that creates an unwritable cache directory for testing permission errors.

2. TestLastFailed

Tests related to the [--last-failed](/projects/286/67385) ([--lf](/projects/286/67298)) and [--failed-first](/projects/286/67385) ([--ff](/projects/286/67298)) options of pytest, which interact closely with the cache to rerun failed tests preferentially.

**Key Features Tested:**

• Use cases of --lf and --ff to rerun failed tests first.

• Behavior when previously failed tests have been fixed or changed.

• Handling of failures during collection or test execution.

• Correct deselection and selection of tests based on cache state.

• Handling of parameterized and xfail tests in failure caching.

• Interaction with nested directories and package structures.

• Behavior when cache is cleared or empty.

• Regression tests for previously reported issues.

**Key Methods:**

• test_lastfailed_usecase(pytester: Pytester, monkeypatch: MonkeyPatch) -> None
  Tests rerunning failed tests and correct cache updates across multiple runs.

• test_failedfirst_order(pytester: Pytester) -> None
  Ensures --ff runs failed tests first.

• test_lastfailed_failedfirst_order(pytester: Pytester) -> None
  Combination of --lf and --ff options.

• test_lastfailed_difference_invocations(...)
  Validates cache behavior across different test files and partial selections.

• Additional tests check edge cases such as non-serializable parameters, skipping collections, handling classes, filtering all tests, and package node interactions.

• get_cached_last_failed(pytester: Pytester) -> list[str]
  Helper method to fetch sorted list of last failed tests from cache.

3. TestNewFirst

Tests for the [--new-first](/projects/286/67223) ([--nf](/projects/286/67298)) option which orders tests so that newer tests (or those with newer modification times) are run first.

**Key Methods:**

• test_newfirst_usecase(pytester: Pytester) -> None
  Tests that --nf runs tests from the newest files first and respects the pytest hook pytest_collection_modifyitems.

• test_newfirst_parametrize(pytester: Pytester) -> None
  Tests --nf behavior with parameterized tests, ensuring parameter sets are ordered correctly.

4. TestReadme

Tests that the README file is created inside the cache directory regardless of test outcomes (pass/fail).

**Key Methods:**

• check_readme(pytester: Pytester) -> bool
  Checks if README.md exists in cache directory.

• test_readme_passed(pytester: Pytester) -> None
  Verifies README creation on passing tests.

• test_readme_failed(pytester: Pytester) -> None
  Verifies README creation on failing tests.

5. Action (Enum)

An enumeration to represent cache directory actions used in parameterized tests.

• MKDIR — For cache directory creation tests.

• SET — For cache value setting tests.

Standalone Test Functions

• test_cache_reportheader(...)
  Tests that pytest prints the correct cache directory path in its report header, including when running in tox environments or with absolute paths.

• test_cache_show(pytester: Pytester) -> None
  Tests the --cache-show CLI option which displays cached values and directories.

• test_gitignore(pytester: Pytester, action: Action) -> None
  Ensures a .gitignore file is automatically created in the cache directory and not overwritten if customized.

• test_preserve_keys_order(pytester: Pytester) -> None
  Validates that dictionary keys order is preserved when saved and retrieved from cache.

• test_does_not_create_boilerplate_in_existing_dirs(pytester: Pytester) -> None
  Checks that .gitignore and README.md are not created when the cache directory is set to an existing directory.

• test_cachedir_tag(pytester: Pytester) -> None
  Verifies the automatic creation of CACHEDIR.TAG file in the cache directory per standard.

• test_clioption_with_cacheshow_and_help(pytester: Pytester) -> None
  Confirms no error occurs when running pytest with --cache-show and --help together.

Important Implementation Details

• The tests use pytest's Pytester utility extensively to create temporary files, directories, and run isolated pytest subprocesses.

• The cache is accessed via config.cache or directly using _pytest.cacheprovider.Cache.for_config() in some tests.

• The cache key naming convention disallows slashes in some contexts and manages hierarchical keys for directory creation.

• Permission tests manipulate filesystem permissions and verify behavior under restricted write access.

• The cache contents are tested for serialization safety (e.g., handling invalid or non-serializable data).

• Tests verify that auxiliary files like .gitignore, README.md, and CACHEDIR.TAG are present to comply with best practices for cache directory management.

• Tests for last-failed and failed-first options verify selective test collection and correct rerunning behavior.

Interaction with Other Parts of the System

• _pytest.cacheprovider.Cache: The core cache API used by pytest for persistent storage of data between test runs. This file tests its interface and behavior.

• pytest Configuration (config): The cache is accessible as an attribute on the pytest config object, and many tests verify this integration.

• CLI options: Tests cover the behavior of cache-related CLI options like --cache-show, --last-failed, --failed-first, and --new-first, ensuring they work as expected with the cache.

• File system: The cache provider creates and manages files and directories under .pytest_cache or custom cache directories, whose creation and permission are tested here.

• pytest hooks: Some tests verify interactions with pytest hooks like pytest_collection_modifyitems to check ordering changes.

Usage Examples

Using Cache in Tests

def test_cachefuncarg(cache):
    val = cache.get("some/thing", None)
    assert val is None
    cache.set("some/thing", [1])
    pytest.raises(TypeError, lambda: cache.get("some/thing"))  # usage without default fails
    val = cache.get("some/thing", [])
    assert val == [1]

Creating Cache Directories

p = config.cache.mkdir("name")
assert p.is_dir()

Customizing Cache Directory

In `pytest.ini`:

[pytest]
cache_dir = custom_cache_dir/subdir

Mermaid Class Diagram

classDiagram
    class TestNewAPI {
        +test_config_cache_mkdir(pytester)
        +test_cache_dir_permissions(pytester)
        +test_config_cache_dataerror(pytester)
        +test_cache_writefail_cachefile_silent(pytester)
        +test_cache_writefail_permissions(unwritable_cache_dir, pytester)
        +test_cache_failure_warns(pytester, monkeypatch, unwritable_cache_dir)
        +test_config_cache(pytester)
        +test_cachefuncarg(pytester)
        +test_custom_rel_cache_dir(pytester)
        +test_custom_abs_cache_dir(pytester, tmp_path_factory)
        +test_custom_cache_dir_with_env_var(pytester, monkeypatch)
        +unwritable_cache_dir(pytester)
    }

    class TestLastFailed {
        +test_lastfailed_usecase(pytester, monkeypatch)
        +test_failedfirst_order(pytester)
        +test_lastfailed_failedfirst_order(pytester)
        +test_lastfailed_difference_invocations(pytester, monkeypatch)
        +test_lastfailed_usecase_splice(pytester, monkeypatch)
        +test_lastfailed_xpass(pytester)
        +test_xfail_not_considered_failure(pytester)
        +test_xfail_strict_considered_failure(pytester)
        +test_failed_changed_to_xfail_or_skip(pytester, mark)
        +test_lf_and_ff_prints_no_needless_message(quiet, opt, pytester)
        +get_cached_last_failed(pytester)
        +test_cache_cumulative(pytester)
        +test_lastfailed_no_failures_behavior_all_passed(pytester)
        +test_lastfailed_no_failures_behavior_empty_cache(pytester)
        +test_lastfailed_skip_collection(pytester)
        +test_lastfailed_skip_collection_with_nesting(pytester)
        +test_lastfailed_with_known_failures_not_being_selected(pytester)
        +test_lastfailed_args_with_deselected(pytester)
        +test_lastfailed_with_class_items(pytester)
        +test_lastfailed_with_all_filtered(pytester)
        +test_packages(pytester)
        +test_non_python_file_skipped(pytester, dummy_yaml_custom_test)
        +test_lastfailed_collectfailure(pytester, monkeypatch)
        +test_lastfailed_failure_subset(pytester, monkeypatch)
        +test_lastfailed_creates_cache_when_needed(pytester)
    }

    class TestNewFirst {
        +test_newfirst_usecase(pytester)
        +test_newfirst_parametrize(pytester)
    }

    class TestReadme {
        +check_readme(pytester)
        +test_readme_passed(pytester)
        +test_readme_failed(pytester)
    }

    class Action {
        <<enumeration>>
        +MKDIR
        +SET
    }

Summary

`test_cacheprovider.py` is a key part of pytest's internal test suite validating the cache provider system. It ensures that the cache API works correctly under a wide range of conditions, integrates properly with pytest features like last-failed and new-first test execution, and manages cache directory contents and permissions properly. The test suite is organized into test classes focusing on related functionalities and uses pytest's own test utilities extensively to verify behavior in realistic scenarios.

This file contributes to pytest's reliability and user experience by preventing regressions and ensuring caching features behave as expected across test runs.