test_pathlib.py

Overview

`test_pathlib.py` is a comprehensive test suite for verifying the functionality of the `_pytest.pathlib` module and related utilities in the pytest testing framework. Its main focus is to ensure correctness, robustness, and compatibility of various path and import-related operations, including:

• Pattern matching of file paths (fnmatch_ex).

• Importing Python modules from paths with different import modes.

• Handling of namespace packages and package resolution.

• Safe filesystem operations such as deleting directories, handling locks, and dealing with long paths.

• Utilities for scanning directories, computing module names, and verifying importability.

• Ensuring that imports behave correctly in complex scenarios involving nested packages, namespace packages, and standard library name clashes.

The tests use pytest fixtures and parametrization extensively to cover a wide range of cases and platforms (notably Windows vs others).

Detailed Explanations

Fixtures

autouse_pytester(pytester: Pytester) -> None

• Purpose: Automatically used fixture to ensure pytester is active for all tests in this module.

• Functionality: Helps restore sys.path to avoid leakage of common module names like test or test.conftest into other tests.

• Usage: Implicitly applied, no direct invocation by tests.

ns_param(request: pytest.FixtureRequest) -> bool

• Purpose: Parameterized fixture to run tests with consider_namespace_packages set to both True and False.

• Usage: Used in tests importing modules with import_path to verify behavior with/without namespace package consideration.

Classes and Their Tests

TestFNMatcherPort

• Purpose: Tests the fnmatch_ex function, a port of py.common.FNMatcher for pattern matching of file paths.

• Key Tests:

  • test_matching: Checks that various glob patterns match expected paths.

  • test_matching_abspath: Ensures absolute paths match relative patterns.

  • test_not_matching: Verifies that non-matching patterns return False.

• Platform Differences: Handles drive letter differences between Windows and Unix-like systems.

TestImportPath

• Purpose: Tests the import_path function and related import mechanisms.

• Fixtures:

  • path1: Sets up a temporary filesystem with Python files and directories to import.

  • preserve_sys: Preserves sys.modules and sys.path around tests to avoid side effects.

  • simple_module: Creates a temporary simple module for import tests.

• Key Tests:

  • test_smoke_test: Basic import of a Python file, verifying module contents.

  • test_import_path_missing_file: Ensures importing a directory without a valid module file raises an error.

  • test_renamed_dir_creates_mismatch: Tests how renaming directories affects import mismatches.

  • test_messy_name, test_dir, test_a, test_b, test_c, test_d: Various tests importing modules and packages, including relative imports inside packages.

  • test_import_after: Imports a module and verifies it is accessible via the Python package.

  • test_check_filepath_consistency: Validates that the module's file matches the expected path, raising ImportPathMismatchError if not.

  • test_ensuresyspath_append: Tests that import with mode "append" appends the root directory to sys.path.

  • test_invalid_path: Importing a non-existing path raises ImportError.

  • test_importmode_importlib: Ensures import mode "importlib" does not modify sys.path and properly imports the module.

  • test_remembers_previous_imports: Checks that repeated imports with "importlib" mode reuse cached modules.

  • test_no_meta_path_found: Tests fallback behavior when no sys.meta_path finders are available.

• Implementation Detail: Uses import_path with different import modes ("append", "importlib", etc.) to test various import scenarios.

TestImportLibMode

• Purpose: Focused tests on the "importlib" import mode.

• Key Tests:

  • Importing modules with dataclasses and pickled objects.

  • Importing modules with the same name but different module paths.

  • Validating module name computation from paths.

  • Testing package imports do not reload init.py multiple times.

  • Testing import of modules with same name as standard library modules.

  • Testing import fallback to normal mechanism when possible.

  • Testing import of submodules and namespace packages with complex layouts.

• Implementation Detail: Uses real file creation and pytest runner (pytester) for integration-like tests.

TestNamespacePackages

• Purpose: Tests support and resolution of namespace packages in import logic.

• Key Methods:

  • setup_directories: Creates a complex namespace package directory structure.

  • Tests for multi-level namespace package imports across different import modes.

  • Tests for proper import rewriting and assertion rewriting in namespace packages.

  • Validates error conditions for incorrect namespace package setups.

  • Tests detection of namespace packages via sys.meta_path.

• Implementation Detail: Uses dynamic module creation and manipulation of sys.path and sys.meta_path.

Standalone Test Functions

• test_resolve_package_path: Verifies resolve_package_path() resolves to package root.

• test_package_unimportable: Tests behavior when package is not importable.

• test_access_denied_during_cleanup: Tests robustness of directory deletion when access denied.

• test_long_path_during_cleanup: Tests handling of long paths, especially on Windows.

• test_get_extended_length_path_str: Tests Windows path extension formatting.

• test_suppress_error_removing_lock: Ensures lock file removal is resilient to errors.

• test_bestrelpath: Validates relative path computation.

• test_commonpath: Validates common path computation.

• test_visit_ignores_errors: Tests directory scanning and error ignoring.

• test_samefile_false_negatives: Windows-specific test to handle os.path.samefile false negatives.

• test_scandir_with_non_existent_directory and test_scandir_handles_os_error: Tests for scandir() utility with error handling.

• test_safe_exists: Tests safe_exists() resilience to OS errors.

• test_import_sets_module_as_attribute*: Tests that imported modules have attributes linking to submodules and parents.

• test_import_submodule_not_namespace: Regression test for submodule import with conflicting directory names.

• test_ns_import_same_name_directory_12592: Regression test for directory names in namespace packages.

• test_is_importable: Tests if modules are importable from given paths.

• test_compute_module_name: Tests computation of module names from file paths.

• validate_namespace_package: Utility to test namespace package import correctness in subprocess.

Important Implementation Details & Algorithms

• Import Modes:

  • "prepend" and "append" modify sys.path to include the root directory accordingly.

  • "importlib" mode uses importlib APIs to import modules without modifying sys.path.

• Namespace Package Resolution:

  • Uses both presence of init.py and sys.meta_path finders to determine if a directory is a namespace package.

• Module Name Computation:

  • Converts file paths to module names by stripping root prefix and replacing path separators with dots.

  • Handles edge cases like init.py files, hidden directories (starting with .), and invalid characters.

• Filesystem Safety:

  • Utilities like maybe_delete_a_numbered_dir and ensure_deletable handle OS errors gracefully.

  • Extended-length paths on Windows are supported via special \\?\ prefix.

• Import Consistency Checks:

  • Detects mismatches between expected module file paths and actual loaded module paths to prevent import errors.

Interaction with Other Parts of the System

• Relies heavily on _pytest.pathlib module — imports many functions and classes from there.

• Uses Pytester and RunResult from _pytest.pytester for integration testing with pytest runs.

• Uses _pytest.tmpdir.TempPathFactory for temporary filesystem fixtures.

• Interacts with Python core modules like importlib, sys, and os.path.

• Tests aim to ensure the _pytest.pathlib module integrates correctly with Python import system and filesystem.

Usage Examples

Importing a Python module from a path considering namespace packages

from _pytest.pathlib import import_path

module = import_path(
    path=Path("/path/to/module.py"),
    root=Path("/path/to"),
    consider_namespace_packages=True,
    mode="importlib"
)
print(module.__name__)

Using fnmatch_ex for pattern matching of paths

from _pytest.pathlib import fnmatch_ex

if fnmatch_ex("tests/**/test_*.py", "tests/unit/test_example.py"):
    print("Matched!")

Mermaid Class Diagram

The file is primarily a test suite containing test classes, so a class diagram is appropriate to show the main test classes and their key methods.

classDiagram
    class TestFNMatcherPort {
        +test_matching(pattern, path)
        +test_matching_abspath()
        +test_not_matching(pattern, path)
    }
    class TestImportPath {
        +setuptestfs(path)
        +test_smoke_test(path1, ns_param)
        +test_import_path_missing_file(path1, ns_param)
        +test_renamed_dir_creates_mismatch(tmp_path, monkeypatch, ns_param)
        +test_messy_name(tmp_path, ns_param)
        +test_dir(tmp_path, ns_param)
        +test_a(path1, ns_param)
        +test_b(path1, ns_param)
        +test_c(path1, ns_param)
        +test_d(path1, ns_param)
        +test_import_after(tmp_path, ns_param)
        +test_check_filepath_consistency(monkeypatch, tmp_path, ns_param)
        +test_ensuresyspath_append(tmp_path, ns_param)
        +test_invalid_path(tmp_path, ns_param)
        +test_importmode_importlib(simple_module, tmp_path, request, ns_param)
        +test_remembers_previous_imports(simple_module, tmp_path, ns_param)
        +test_no_meta_path_found(simple_module, monkeypatch, tmp_path, ns_param)
    }
    class TestImportLibMode {
        +test_importmode_importlib_with_dataclass(tmp_path, ns_param)
        +test_importmode_importlib_with_pickle(tmp_path, ns_param)
        +test_importmode_importlib_with_pickle_separate_modules(tmp_path, ns_param)
        +test_module_name_from_path(tmp_path)
        +test_resolve_pkg_root_and_module_name(tmp_path, monkeypatch, pytester)
        +test_insert_missing_modules(monkeypatch, tmp_path)
        +test_import_module_using_spec(b_is_package, insert_modules, tmp_path)
        +test_parent_contains_child_module_attribute(monkeypatch, tmp_path)
        +test_importlib_package(monkeypatch, tmp_path, ns_param)
        +test_importlib_root_is_package(pytester)
        +test_importlib_same_name_as_stl(pytester, ns_param, tmp_path, name)
        +test_import_using_normal_mechanism_first(monkeypatch, pytester, ns_param)
        +test_import_using_normal_mechanism_first_integration(monkeypatch, pytester, ns_param)
        +test_import_path_imports_correct_file(pytester, ns_param)
    }
    class TestNamespacePackages {
        +setup_directories(tmp_path, monkeypatch, pytester)
        +test_resolve_pkg_root_and_module_name_ns_multiple_levels(tmp_path, monkeypatch, pytester, import_mode)
        +test_ns_multiple_levels_import_rewrite_assertions(tmp_path, monkeypatch, pytester)
        +test_ns_multiple_levels_import_error(tmp_path, pytester)
        +test_incorrect_namespace_package(tmp_path, monkeypatch, pytester, import_mode)
        +test_detect_meta_path(tmp_path, monkeypatch, pytester)
        +test_full_ns_packages_without_init_files(pytester, tmp_path, monkeypatch, insert)
    }

Summary

This test module rigorously validates pytest's pathlib and import mechanisms, including complex scenarios like namespace packages, import modes, and filesystem edge cases. It helps maintain the reliability of pytest's internal import utilities and path abstractions critical for test discovery and execution.