tmpdir.py

Overview

The [tmpdir.py](/projects/286/67435) file provides robust support for creating and managing temporary directories for test functions within the pytest testing framework. It primarily defines the `TempPathFactory` class that centralizes temporary directory creation under a common base directory and enforces retention policies for cleanup or preservation of these directories after test execution.

This module integrates deeply with pytest’s fixture and hook systems to:

• Provide unique temporary directories isolated per test function.

• Manage a configurable base directory for all temporary directories.

• Apply configurable retention policies (all, failed, none) to control the lifecycle of temporary directories.

• Clean up temporary directories automatically based on test outcomes and user preferences.

• Ensure security by verifying ownership and permissions of temporary directory roots.

The file exposes session-scoped and function-scoped fixtures (`tmp_path_factory` and `tmp_path`) to deliver temporary directory paths to test code, while managing lifecycle and cleanup transparently.

Detailed Explanation of Classes, Functions, and Methods

Class: TempPathFactory

A [@dataclass](/projects/286/67337) and [@final](/projects/286/67223) class responsible for creating and managing temporary directories under a shared base temporary directory. It encapsulates directory creation, base directory management, and retention policy enforcement.

Attributes

• _given_basetemp: Path | None
  User-supplied base temporary directory, if any.

• _trace: Any
  Trace function for logging directory creation events (used internally by pytest).

• _basetemp: Path | None
  Cached path to the resolved base temporary directory.

• _retention_count: int
  Number of temporary directory sessions to retain.

• _retention_policy: RetentionType ("all" | "failed" | "none")
  Policy controlling which temporary directories are kept after test runs.

Methods

__init__

def __init__(
    self,
    given_basetemp: Path | None,
    retention_count: int,
    retention_policy: RetentionType,
    trace,
    basetemp: Path | None = None,
    *,
    _ispytest: bool = False,
) -> None:

• Purpose: Initialize the factory with optional user base temp directory, retention parameters, and tracing.

• Parameters:

  • given_basetemp: Optional user-specified base directory path.

  • retention_count: Number of past temp directories to retain.

  • retention_policy: Retention policy string (all, failed, none).

  • trace: Internal trace logging callable (pytest internal).

  • basetemp: Optional pre-computed base temp directory path.

  • _ispytest: Internal flag ensuring this is called from pytest.

• Behavior: Normalizes and stores paths, sets retention policies, checks internal usage.

from_config

@classmethod
def from_config(cls, config: Config, *, _ispytest: bool = False) -> TempPathFactory:

• Purpose: Factory constructor using pytest configuration options.

• Parameters:

  • config: pytest Config object.

  • _ispytest: Internal flag for pytest usage check.

• Returns: An instance of TempPathFactory.

• Usage:

factory = TempPathFactory.from_config(config)

• Details: Reads tmp_path_retention_count and tmp_path_retention_policy from pytest ini options, validates them, and initializes the factory accordingly.

_ensure_relative_to_basetemp

def _ensure_relative_to_basetemp(self, basename: str) -> str:

• Purpose: Validate that a given directory name (basename) is a normalized relative path within the base temp directory (no path traversal).

• Parameters:
  basename - The relative directory name requested.

• Returns: Normalized relative basename.

• Raises: ValueError if basename is not a normalized relative path.

• Usage: Used internally before creating any temp directory.

mktemp

def mktemp(self, basename: str, numbered: bool = True) -> Path:

• Purpose: Create a new temporary directory under the base directory, optionally with a unique numbered suffix.

• Parameters:

  • basename: Base name for the directory (must be relative).

  • numbered: If True, append a unique incremental number suffix (e.g., basename-0, basename-1).

• Returns: The Path to the newly created directory.

• Usage Example:

temp_dir = factory.mktemp("testdata-", numbered=True)

• Notes:

  • If numbered=False, creates exactly the given directory name.

  • Uses make_numbered_dir helper to guarantee unique directory names.

  • Traces directory creation internally.

getbasetemp

def getbasetemp(self) -> Path:

• Purpose: Lazily determine and create the base temporary directory, ensuring it exists and has proper permissions.

• Returns: The base Path for temporary directories.

• Usage: Called automatically when creating new temp directories or accessed externally.

• Implementation Details:

  • If user-specified base temp exists, it is wiped and recreated.

  • Otherwise, creates a base directory under system temp (tempfile.gettempdir()), scoped by username.

  • Verifies ownership and permissions for security.

  • Uses make_numbered_dir_with_cleanup to create a numbered base directory with cleanup of old directories according to retention count.

  • Caches result in _basetemp.

  • Traces creation event.

Function: get_user

def get_user() -> str | None:

• Purpose: Return the current system username or None if it cannot be determined.

• Usage: Used internally to create user-scoped temp directories.

• Implementation: Uses getpass.getuser() but safely catches import errors or OS-related errors.

Pytest Hooks and Fixtures

pytest_configure

def pytest_configure(config: Config) -> None:

• Purpose: Pytest hook called during pytest configuration phase.

• Behavior:

  • Creates a TempPathFactory instance from pytest config.

  • Attaches the factory instance to the config object as _tmp_path_factory.

  • Uses MonkeyPatch to ensure cleanup of the attribute on config teardown.

• Effect: Makes the temporary directory factory available globally via pytest config.

pytest_addoption

def pytest_addoption(parser: Parser) -> None:

• Purpose: Register ini options related to temporary directory retention.

• Options Added:

  • tmp_path_retention_count: Number of temp directory sessions to keep (default 3).

  • tmp_path_retention_policy: Retention policy (all, failed, none) (default all).

Fixture: tmp_path_factory

@fixture(scope="session")
def tmp_path_factory(request: FixtureRequest) -> TempPathFactory:

• Purpose: Provide a session-scoped fixture for the TempPathFactory.

• Returns: The TempPathFactory instance attached to pytest config.

• Usage: Inject this fixture into other fixtures or tests to create temporary directories.

Internal Helper: _mk_tmp

def _mk_tmp(request: FixtureRequest, factory: TempPathFactory) -> Path:

• Purpose: Generate a safe, sanitized temporary directory name based on the test node name and create it using the factory.

• Parameters:

  • request: Fixture request object containing test metadata.

  • factory: The TempPathFactory instance.

• Returns: The newly created temporary directory Path.

• Behavior:

  • Sanitizes test node name by replacing non-word characters with underscores.

  • Truncates name to 30 chars.

  • Calls factory.mktemp with the sanitized name and numbered=True.

Fixture: tmp_path

@fixture
def tmp_path(request: FixtureRequest, tmp_path_factory: TempPathFactory) -> Generator[Path]:

• Purpose: Provide a function-scoped temporary directory unique to each test function invocation.

• Returns: A pathlib.Path object pointing to the temp directory.

• Behavior:

  • Creates a temp directory using _mk_tmp.

  • Yields the path to the test.

  • After test finishes, checks retention policy:

    • If policy is "failed" and the test passed, removes the directory.

  • Cleans up stash keys related to temp path results.

• Usage Example:

def test_something(tmp_path):
    file = tmp_path / "data.txt"
    file.write_text("hello")
    assert file.exists()

Hook: pytest_sessionfinish

def pytest_sessionfinish(session, exitstatus: int | ExitCode):

• Purpose: Cleanup after the entire test session finishes.

• Behavior:

  • If all tests passed (exitstatus == 0) and retention policy is "failed", and no user base temp was provided:

    • Removes the entire base temporary directory.

  • Regardless, cleans up dead symbolic links under the base temp directory.

• Effect: Maintains disk hygiene by removing stale temporary directories and symlinks.

Hook: pytest_runtest_makereport

@hookimpl(wrapper=True, tryfirst=True)
def pytest_runtest_makereport(item: Item, call) -> Generator[None, TestReport, TestReport]:

• Purpose: Capture test phase results (setup, call, teardown) to track test success or failure.

• Behavior:

  • Stores boolean pass/fail status for each phase in the stash of the test item keyed by tmppath_result_key.

  • Enables the tmp_path fixture to decide retention based on test outcome.

• Usage: Internal hook, called automatically by pytest.

Important Implementation Details and Algorithms

• Unique Directory Naming:
  Uses make_numbered_dir and make_numbered_dir_with_cleanup helpers to create directories with unique incremental suffixes to avoid conflicts when tests are run concurrently or multiple temp directories are created with the same base name.

• Base Directory Security:
  When creating user-scoped base directories under system temp, the factory verifies ownership (st_uid) and restricts permissions to prevent security risks in shared temp directories.

• Retention Policy Enforcement:
  Retention policies are enforced both at the per-test level (in tmp_path fixture) and at session end (in pytest_sessionfinish), ensuring both fine-grained and coarse-grained cleanup.

• Stash Usage for Test Outcomes:
  The use of request.node.stash keyed by a StashKey allows storing arbitrary information related to test outcomes without polluting the namespace, providing a clean mechanism to track test pass/fail per phase.

• MonkeyPatch for Config Attribute:
  Attaching the factory to the pytest config via MonkeyPatch ensures cleanup of the attribute on test session teardown, avoiding side effects or memory leaks.

• Path Normalization and Safety Checks:
  _ensure_relative_to_basetemp ensures that directory names passed to the factory cannot escape the base directory, mitigating security concerns related to path traversal.

Interaction with Other Parts of the System

• pytest Configuration and CLI:
  Reads configuration from pytest ini options (tmp_path_retention_count and tmp_path_retention_policy), and command-line options (basetemp) to customize behavior.

• pytest Fixtures:
  Exposes tmp_path_factory and tmp_path fixtures that downstream plugins or tests can inject for temporary directory usage.

• pytest Hooks:
  Uses hooks (pytest_configure, pytest_runtest_makereport, pytest_sessionfinish) to manage lifecycle and cleanup according to test execution phases and results.

• Internal Pytest Utilities:
  Relies on internal utilities from _pytest.pathlib for directory creation and cleanup (make_numbered_dir, rm_rf, cleanup_dead_symlinks).

• MonkeyPatch:
  Uses pytest's MonkeyPatch utility to safely attach and remove attributes from the config object during the test lifecycle.

• Test Reporting:
  Uses TestReport objects to track test phase results and decide whether to retain or remove temporary directories.

Visual Diagram: Class Structure and Main Methods

classDiagram
    class TempPathFactory {
        -_given_basetemp: Path | None
        -_trace: Any
        -_basetemp: Path | None
        -_retention_count: int
        -_retention_policy: RetentionType
        +__init__(given_basetemp, retention_count, retention_policy, trace, basetemp=None, _ispytest=False)
        +from_config(config, _ispytest=False) TempPathFactory
        -_ensure_relative_to_basetemp(basename) str
        +mktemp(basename, numbered=True) Path
        +getbasetemp() Path
    }

    TempPathFactory ..> "uses" make_numbered_dir
    TempPathFactory ..> "uses" make_numbered_dir_with_cleanup
    TempPathFactory ..> "uses" rm_rf

Summary

The [tmpdir.py](/projects/286/67435) file implements a comprehensive temporary directory management system for pytest tests, providing:

• A configurable factory (TempPathFactory) to create safe, isolated temporary directories.

• Fixtures (tmp_path_factory, tmp_path) to inject these directories into tests.

• Retention policies to control cleanup behavior based on test outcomes.

• Security checks to ensure safe temp directory usage.

• Lifecycle hooks for automated cleanup after tests and sessions.

This design ensures reproducible, isolated test environments with minimal manual cleanup, improving test reliability and developer experience.

End of Documentation for tmpdir.py