Temporary Directory Management

This module provides fixtures and utilities for creating and managing temporary directories unique to each test function invocation. It addresses the need for isolated, disposable filesystem locations during test runs, ensuring tests do not interfere with each other's data and enabling configurable retention policies for debugging or cleanup purposes.

Core Concepts and Purpose

Temporary directories are essential in testing scenarios where tests need to write files, logs, or other artifacts without polluting the global filesystem or affecting other tests. This module:

• Provides a centralized factory (TempPathFactory) to create and manage temporary directories under a common base directory.

• Offers a per-test temporary directory fixture (tmp_path) that guarantees isolation.

• Supports configurable retention policies to control how long temporary directories persist after test execution, facilitating debugging of failed tests or automatic cleanup.

• Ensures secure and unique directory creation with proper permissions and naming conventions.

• Integrates with pytest's lifecycle through hooks for setup, reporting, and teardown to manage these directories effectively.

How It Works

TempPathFactory: The Central Factory

`TempPathFactory` is a dataclass responsible for managing the lifecycle and creation of temporary directories within a shared base directory (`basetemp`).

• Base Directory Management: It determines or creates the base temporary directory where all test-specific temporary directories reside. This can be user-specified or automatically generated under system temp paths.

• Safe Directory Creation: It creates uniquely numbered subdirectories to avoid conflicts. The mktemp method either creates a directory with a suffix number ensuring uniqueness or a single named directory.

• Retention Policies: It uses configuration options to decide how many previous temporary directories to keep (tmp_path_retention_count) and under what conditions (tmp_path_retention_policy). The policies include:

  • "all": Keep all temporary directories.

  • "failed": Keep only directories from failed tests.

  • "none": Do not keep any directories after tests finish.

• Security and Ownership Checks: When auto-creating base directories, it verifies ownership and permissions to avoid unsafe shared temp directories, preventing potential security issues.

• Integration with pytest Config: A TempPathFactory instance is created during pytest configuration (pytest_configure) and attached to the config object for later use by fixtures.

Per-Test Temporary Directory Fixture (tmp_path)

• Each test function requesting the tmp_path fixture receives a unique temporary directory created by the factory.

• The directory name is derived from the test node's name, sanitized and truncated to a safe length.

• After the test runs, the fixture inspects the test outcome and the configured retention policy to decide whether to remove the directory:

  • If the policy is "failed" and the test passed, the directory is removed.

  • For other policies, retention behavior follows the configuration.

• This ensures that temporary directories for failed tests can be preserved for post-mortem analysis.

Lifecycle and Cleanup Hooks

• pytest_sessionfinish hook runs after the entire test session and performs cleanup:

  • Removes the whole base temporary directory if all tests passed and the policy is "failed", and the base directory was not explicitly specified by the user.

  • Cleans up any dead symlinks in the base temporary directory to maintain hygiene.

• pytest_runtest_makereport hook stores test phase results to determine whether directories from passed or failed tests should be retained or removed.

Interaction with Other System Parts

• Configuration System: Reads ini options tmp_path_retention_count and tmp_path_retention_policy to configure behavior. These options are registered in pytest_addoption.

• Fixture System: Implements tmp_path_factory as a session-scoped fixture providing access to the factory instance, and tmp_path as a function-scoped fixture for per-test temporary directories.

• Test Reporting: Uses test outcome information (TestReport) collected during test execution phases to decide retention of directories.

• Path Utilities: Uses helper functions like make_numbered_dir_with_cleanup, rm_rf (recursive remove), and cleanup_dead_symlinks from src/_pytest/pathlib.py to manage directory creation and cleanup safely.

• MonkeyPatch Utility: Uses MonkeyPatch to attach the factory instance to the pytest config object during configuration for internal access.

Important Concepts and Design Patterns

• Factory Pattern: TempPathFactory encapsulates the logic for creating temporary paths, abstracting directory naming, uniqueness, and base directory management.

• Fixture Injection: Uses fixtures to provide temporary directory paths to test functions, leveraging pytest's fixture dependency injection and scopes.

• Config-Driven Behavior: Retention policies and counts are configurable via user-defined ini options, allowing users to tune temporary directory cleanup behavior based on their needs.

• Lifecycle Hooks for Resource Management: Employs pytest hooks (pytest_configure, pytest_sessionfinish, and pytest_runtest_makereport) to manage lifecycle events of temporary directories, ensuring proper creation and cleanup aligned with test execution phases.

• Security Considerations: Checks user ownership and permissions on shared temp directories to avoid security issues in multi-user environments.

Code References and Examples

Creation of TempPathFactory from pytest config

@classmethod
def from_config(cls, config: Config, *, _ispytest: bool = False) -> TempPathFactory:
    count = int(config.getini("tmp_path_retention_count"))
    policy = config.getini("tmp_path_retention_policy")
    return cls(
        given_basetemp=config.option.basetemp,
        retention_count=count,
        retention_policy=policy,
        trace=config.trace.get("tmpdir"),
        _ispytest=True,
    )

This snippet shows how the factory is configured with retention parameters and optionally a user-defined base temp directory.

Fixture Providing Per-Test Temporary Directory

@fixture
def tmp_path(request: FixtureRequest, tmp_path_factory: TempPathFactory) -> Generator[Path]:
    path = _mk_tmp(request, tmp_path_factory)
    yield path

    policy = tmp_path_factory._retention_policy
    result_dict = request.node.stash[tmppath_result_key]

    if policy == "failed" and result_dict.get("call", True):
        rmtree(path, ignore_errors=True)

    del request.node.stash[tmppath_result_key]

This fixture creates a unique temp directory per test, yields it, and cleans up after test execution according to the retention policy.

Cleanup After Test Session

def pytest_sessionfinish(session, exitstatus: int | ExitCode):
    tmp_path_factory: TempPathFactory = session.config._tmp_path_factory
    basetemp = tmp_path_factory._basetemp
    if basetemp is None:
        return

    policy = tmp_path_factory._retention_policy
    if (
        exitstatus == 0
        and policy == "failed"
        and tmp_path_factory._given_basetemp is None
    ):
        if basetemp.is_dir():
            rmtree(basetemp, ignore_errors=True)

    if basetemp.is_dir():
        cleanup_dead_symlinks(basetemp)

This hook removes the base temp directory if all tests passed and retention policy dictates so, plus it cleans up dead symbolic links.

Mermaid Diagram: Temporary Directory Management Workflow

flowchart TD
    Start[Start Test Session]
    Config[Load tmp_path_retention_count & tmp_path_retention_policy]
    CreateFactory[Create TempPathFactory]
    RegisterFactory[Attach Factory to Config]
    ForEachTest[Test Function Start]
    TmpDir[Create Unique Temp Directory via Factory]
    RunTest[Test Execution]
    StoreResult[Store Test Outcome]
    CleanupTmpDir{Retention Policy?}
    RemoveDir[Remove Temp Directory]
    KeepDir[Keep Temp Directory]
    NextTest[Next Test or Session Finish]
    SessionFinish[After Session Finish]
    CleanupBaseDir{Retention Policy & Outcome}
    RemoveBaseDir[Remove Base Temp Directory]
    KeepBaseDir[Keep Base Temp Directory]
    CleanupSymlinks[Cleanup Dead Symlinks]

    Start --> Config --> CreateFactory --> RegisterFactory
    RegisterFactory --> ForEachTest
    ForEachTest --> TmpDir --> RunTest --> StoreResult --> CleanupTmpDir
    CleanupTmpDir -->|Remove if passed & policy=failed| RemoveDir
    CleanupTmpDir -->|Otherwise| KeepDir
    RemoveDir --> NextTest
    KeepDir --> NextTest
    NextTest -->|More Tests| ForEachTest
    NextTest -->|No More Tests| SessionFinish
    SessionFinish --> CleanupBaseDir
    CleanupBaseDir -->|Remove if all passed & policy=failed| RemoveBaseDir
    CleanupBaseDir -->|Otherwise| KeepBaseDir
    RemoveBaseDir --> CleanupSymlinks
    KeepBaseDir --> CleanupSymlinks

This documentation explains the temporary directory management system's role, behavior, and integration within pytest, focusing on providing isolated, configurable, and secure filesystem locations for test executions.