Retention and Cleanup

Purpose

Temporary directories created during test runs can accumulate quickly, consuming disk space and potentially cluttering the environment. The **Retention and Cleanup** subtopic addresses this by providing configurable policies that determine when these temporary directories should be preserved or removed. This feature allows developers to retain directories selectively—for example, keeping only those from failed test runs for troubleshooting—while cleaning up others automatically to maintain a tidy workspace.

Functionality

This subtopic introduces a retention policy mechanism that governs the lifecycle of temporary directories under the `tmp_path` fixture. The core functionality includes:

• Retention Policy Configuration: Users configure how many temporary directories to retain and under what conditions via two settings:

  • tmp_path_retention_policy: Defines which directories to keep. Options include:

    • "all": Retain directories for all tests.

    • "failed": Retain directories only for failed tests.

    • "none": Remove all directories after test completion.

  • tmp_path_retention_count: Specifies how many past temporary directory sessions to retain before cleanup.

• Selective Cleanup Based on Test Outcome: After each test, the system evaluates the retention policy and test result. For example, if the policy is "failed" and the test passed, the temporary directory is deleted immediately to save space.

• Session-Level Cleanup: At the end of a test session, if all tests passed and the retention policy is "failed", the entire base temporary directory (containing all session temp dirs) is removed unless it was explicitly set by the user.

• Safe and Best-Effort Cleanup: Cleanup operations use best-effort approaches to avoid interference from permission issues or lingering resources. Dead symbolic links inside the base temporary directory are also cleaned up after the session finishes.

• Integration with Pytest Hooks and Fixtures:

  • The tmp_path_factory session fixture provides a TempPathFactory instance configured with the retention policy.

  • The tmp_path function-scoped fixture creates unique temporary directories per test and applies retention logic on test completion.

  • The pytest_runtest_makereport hook tracks test outcomes to decide retention per test.

  • The pytest_sessionfinish hook performs cleanup at the end of the test session.

Key Code Interactions

• The TempPathFactory class encapsulates retention settings and manages creation and cleanup of temporary directories:

class TempPathFactory:
    def __init__(self, *, retention_count: int, retention_policy: RetentionType, ...):
        self._retention_count = retention_count
        self._retention_policy = retention_policy
    ...
    def mktemp(self, basename: str, numbered: bool = True) -> Path:
        # Creates a uniquely numbered temp directory.

• The tmp_path fixture creates a temp directory and cleans it up conditionally after test execution:

@fixture
def tmp_path(request, tmp_path_factory):
    path = tmp_path_factory.mktemp(...)
    yield path
    if tmp_path_factory._retention_policy == "failed" and test_passed:
        rmtree(path, ignore_errors=True)

• The hook pytest_runtest_makereport captures test pass/fail status to inform retention decisions:

@hookimpl(wrapper=True, tryfirst=True)
def pytest_runtest_makereport(item, call):
    rep = yield
    item.stash.setdefault(tmppath_result_key, {})[rep.when] = rep.passed
    return rep

• The hook pytest_sessionfinish cleans up the base temporary directory if appropriate:

def pytest_sessionfinish(session, exitstatus):
    if exitstatus == 0 and policy == "failed" and no_given_basetemp:
        rmtree(basetemp, ignore_errors=True)
    cleanup_dead_symlinks(basetemp)

Integration

Retention and Cleanup tightly integrate with the **Temporary Directory Management** parent topic by extending the lifecycle management of temporary directories beyond simple creation and deletion. While the parent topic focuses on providing unique temporary directories per test, this subtopic adds intelligent policies that conditionally preserve or remove these directories based on test outcomes and user preferences.

This subtopic also interacts with:

• Test Execution and Reporting: By consuming test result data to implement conditional cleanup.

• Plugin System and Hooks: Leveraging pytest's hook system (pytest_runtest_makereport, pytest_sessionfinish) to track test outcomes and trigger cleanup actions.

• Fixture Management: Providing the tmp_path fixture with enhanced retention behavior without altering the fixture's core interface.

Together, these integrations ensure that temporary directories are managed efficiently, aiding both debugging (by retaining relevant dirs) and resource hygiene (by cleaning up unnecessary data).

Diagram

flowchart TD
    Start[Test Starts]
    MkTmp[Create Temp Dir (mktemp)]
    RunTest[Test Execution]
    CaptureResult[Capture Test Result]
    CheckPolicy{Retention Policy}
    Passed?{Test Passed?}
    CleanupDir[Delete Temp Dir]
    KeepDir[Keep Temp Dir]
    End[Test Ends]

    Start --> MkTmp --> RunTest --> CaptureResult --> CheckPolicy
    CheckPolicy -->|none| CleanupDir
    CheckPolicy -->|all| KeepDir
    CheckPolicy -->|failed| Passed?
    Passed? -->|Yes| CleanupDir
    Passed? -->|No| KeepDir
    CleanupDir --> End
    KeepDir --> End

This flowchart visualizes the decision process for retaining or cleaning up temporary directories based on the configured retention policy and the outcome of each test.