collect.py

Overview

The [collect.py](/projects/286/67331) file contains a comprehensive suite of tests primarily targeting **pytest's collection mechanism** for test modules, test classes, and test functions. The file validates how pytest discovers, collects, and organizes test items from Python source files, ensuring correct behavior across a wide variety of scenarios including syntax errors, import errors, parametrization, custom collection hooks, and edge cases like duplicate test names or special Python constructs.

The tests in this file leverage pytest's own testing utilities (`pytester`, `MonkeyPatch`, `Pytester`, etc.) to simulate and verify the behavior of pytest internals during test collection, parametrization, and reporting phases.

In essence, this file acts as a **meta-test suite** for pytest's test collection subsystem, verifying both correctness and robustness against regressions.

Detailed Explanation of Classes and Functions

Class: TestModule

Tests related to the collection of **test modules** (Python files interpreted as test containers).

Methods:

• test_failing_import(self, pytester: Pytester) -> None

  • Purpose: Verifies that attempting to collect a module with an invalid import name raises a Collector.CollectError.

  • Parameters:

    • pytester: pytest's testing utility to create test files and collect items.

  • Usage:

    pytester.getmodulecol("import invalid_module_name")
    

  • Raises: Collector.CollectError when .collect() is called.

• test_import_duplicate(self, pytester: Pytester) -> None

  • Purpose: Checks that pytest warns about import mismatches when two test files with the same module name exist in different directories.

  • Details: Creates two directories each containing test_whatever.py and verifies that pytest detects and warns about the import mismatch.

• test_import_prepend_append(self, pytester: Pytester, monkeypatch: MonkeyPatch) -> None

  • Purpose: Validates pytest's behavior with import modes prepend and append for sys.path during test import.

  • Details: Creates two roots with modules named the same and tests import modes via monkeypatching sys.path.

• test_syntax_error_in_module(self, pytester: Pytester) -> None

  • Purpose: Asserts that syntax errors in test modules raise a collection error.

• test_module_considers_pluginmanager_at_import(self, pytester: Pytester) -> None

  • Purpose: Ensures that pytest plugins declared in test modules are considered during import, raising ImportError if invalid.

• test_invalid_test_module_name(self, pytester: Pytester) -> None

  • Purpose: Verifies pytest's error reporting when test modules have invalid Python names (e.g., containing dots).

• test_show_traceback_import_error(self, pytester: Pytester, verbose: int) -> None

  • Purpose: Tests that import errors during module collection display tracebacks properly based on verbosity level.

  • Parameters:

    • verbose: Controls verbosity of pytest output.

  • Details: Ensures internal pytest frames are omitted at low verbosity but shown at high verbosity.

• test_show_traceback_import_error_unicode(self, pytester: Pytester) -> None

  • Purpose: Checks that import errors with Unicode error messages are displayed properly.

Class: TestClass

Tests related to the collection of **test classes**.

Methods:

• test_class_with_init_warning(self, pytester: Pytester) -> None

  • Purpose: Verifies pytest warns when collecting classes with __init__ methods, which it cannot collect as test classes.

• test_class_with_new_warning(self, pytester: Pytester) -> None

  • Purpose: Similar to above but for classes defining __new__.

• test_class_subclassobject(self, pytester: Pytester) -> None

  • Purpose: Ensures that classes subclassing object without test methods are collected as empty (0 items).

• test_static_method(self, pytester: Pytester) -> None

  • Purpose: Confirms support for collecting @staticmethod test methods.

• test_setup_teardown_class_as_classmethod(self, pytester: Pytester) -> None

  • Purpose: Tests that class-level setup and teardown methods defined as @classmethod work correctly.

• test_issue1035_obj_has_getattr(self, pytester: Pytester) -> None

  • Purpose: Checks that objects with a __getattr__ method do not cause unexpected collection.

• test_issue1579_namedtuple(self, pytester: Pytester) -> None

  • Purpose: Tests that namedtuple-based classes with a __new__ method are not collected as test classes.

• test_issue2234_property(self, pytester: Pytester) -> None

  • Purpose: Ensures properties raising exceptions do not break test collection.

• test_does_not_discover_properties(self, pytester: Pytester) -> None

  • Purpose: Regression test ensuring properties are not mistakenly collected as tests.

• test_does_not_discover_instance_descriptors(self, pytester: Pytester) -> None

  • Purpose: Similar to above but for descriptors acting like properties (e.g., cached_property).

• test_abstract_class_is_not_collected(self, pytester: Pytester) -> None

  • Purpose: Verifies that abstract classes (with abstract methods) are not collected as tests.

Class: TestFunction

Tests related to the collection and behavior of **test functions**.

Methods:

• test_getmodulecollector(self, pytester: Pytester) -> None

  • Purpose: Verifies the parent collector of a test function is a pytest Module.

• test_function_as_object_instance_ignored(self, pytester: Pytester) -> None

  • Purpose: Ensures callable instances that are not functions are ignored by pytest.

• make_function(pytester: Pytester, **kwargs: Any) -> Any

  • Purpose: Utility method to create pytest Function instances for tests.

  • Parameters:

    • kwargs: Keyword arguments forwarded to pytest.Function.

  • Returns: pytest Function instance.

• test_function_equality(self, pytester: Pytester) -> None

  • Purpose: Tests equality comparisons of pytest Function instances.

• test_repr_produces_actual_test_id(self, pytester: Pytester) -> None

  • Purpose: Checks that the string representation (repr) of a pytest Function reflects the test id properly.

• Parametrization tests: A large set of tests verifying the correct behavior of @pytest.mark.parametrize with various arguments, including empty sets, tuples, non-hashable values, nested parametrization, skip/xfail marks, and overrides of fixtures.

• test_pyfunc_call(self, pytester: Pytester) -> None

  • Purpose: Tests that the pytest hook pytest_pyfunc_call works correctly with plugins overriding function call behavior.

• test_multiple_parametrize(self, pytester: Pytester) -> None

  • Purpose: Verifies multiple nested parametrize decorators produce the expected cartesian product of test cases.

• test_function_originalname(self, pytester: Pytester) -> None

  • Purpose: Checks that the original function name is correctly preserved across parametrized tests.

• test_function_with_square_brackets(self, pytester: Pytester) -> None

  • Purpose: Ensures functions with square brackets in their names do not cause errors during collection or execution.

Class: TestSorting

Tests related to sorting and comparison behaviors of pytest test items.

• test_check_equality(self, pytester: Pytester) -> None

  • Tests == and != behavior between pytest Function instances and other objects.

• test_allow_sane_sorting_for_decorators(self, pytester: Pytester) -> None

  • Ensures decorated functions with place_as attribute sort in a stable and expected order.

• test_ordered_by_definition_order(self, pytester: Pytester) -> None

  • Verifies that test functions are collected in the order defined in source files, even across inheritance.

Class: TestConftestCustomization

Tests for pytest hooks related to customizing the **collection** process via `conftest.py` files.

• test_pytest_pycollect_module(self, pytester: Pytester) -> None

  • Tests the pytest_pycollect_makemodule hook to override module collection.

• test_customized_pymakemodule_issue205_subdir(self, pytester: Pytester) -> None

  • Tests that a wrapper hook can modify module objects on collection.

• test_customized_pymakeitem(self, pytester: Pytester) -> None

  • Verifies the pytest_pycollect_makeitem hook to modify test item objects after creation.

• test_pytest_pycollect_makeitem(self, pytester: Pytester) -> None

  • Tests overriding item creation for specific function names.

• test_issue2369_collect_module_fileext(self, pytester: Pytester) -> None

  • Demonstrates collecting files with unusual extensions (e.g., .narf) as Python modules using meta-path import hooks.

• test_early_ignored_attributes(self, pytester: Pytester) -> None

  • Validates that certain built-in attributes are ignored early to optimize collection.

Other Functions and Tests

• test_setup_only_available_in_subdir(pytester: Pytester) -> None

  • Tests that setup/teardown hooks in subdirectories only apply to tests in those subdirectories.

• test_modulecol_roundtrip(pytester: Pytester) -> None

  • Verifies that a collected module can be re-collected by nodeid without loss.

• TestTracebackCutting

  • Contains tests for traceback filtering and formatting during skips, import errors, and fixture failures.

• TestReportInfo

  • Tests for the .reportinfo() method on test items to provide accurate source location and messages.

• Various standalone tests validate:

  • Customization of python test discovery patterns.

  • Handling of unorderable types or invalid test callables.

  • Collection behavior in package hierarchies.

  • Prevention of infinite recursion during package collection.

  • Behavior with duplicate test directories.

  • Regression cases for syntax errors, import errors, and collection warnings.

Important Implementation Details / Algorithms

• Use of pytest's pytester and Pytester fixtures: These are leveraged to dynamically create test files, directories, and run pytest commands in-process to validate collection behavior.

• Testing import modes (prepend, append) to verify sys.path manipulation during test import.

• Parametrization testing: The file extensively tests @pytest.mark.parametrize behavior with various edge cases, such as empty sets, indirect parametrization, and interaction with fixtures.

• Custom collection hooks: Tests confirm that pytest hooks like pytest_pycollect_makemodule and pytest_pycollect_makeitem can be used to alter the collection process and that these customizations are respected.

• Traceback filtering: Tests ensure that pytest correctly filters out internal frames and dynamically generated code frames from tracebacks shown to users.

• Ordering and sorting: Tests verify that the order of collected items respects definition order and that decorated functions sort in a stable manner.

• Handling of non-standard Python constructs: The tests cover cases like properties, descriptors, abstract classes, namedtuples, and classes with __init__ or __new__ to ensure pytest's collection logic behaves correctly.

Interaction with Other Parts of the System

• This file interacts deeply with pytest internals:

  • pytest.Function, pytest.Module, and pytest.Class classes representing test items.

  • The collection infrastructure that discovers test files, modules, classes, and functions.

  • Collection hooks (pytest_pycollect_makemodule, pytest_pycollect_makeitem) allowing customization of collection.

  • The test execution engine indirectly: tests verify how collected tests are run and reported.

  • The traceback and error reporting system (e.g., _pytest._code.Traceback) for filtering tracebacks.

  • Fixture management and parametrization mechanisms to ensure correct test parameter injection.

• The file depends on pytest's test support APIs (pytester, Pytester) to simulate and run test collection scenarios.

• It also manipulates the filesystem (creating files/directories) and sys.modules/sys.path during some tests to simulate various import scenarios and file layouts.

Usage Examples

The file itself is a test suite and not intended for direct usage beyond pytest's internal test runs. However, examples of how tests are written in this file:

def test_class_with_init_warning(self, pytester: Pytester) -> None:
    pytester.makepyfile(
        """
        class TestClass1(object):
            def __init__(self):
                pass
    """
    )
    result = pytester.runpytest()
    result.stdout.fnmatch_lines(
        [
            "*cannot collect test class 'TestClass1' because it has "
            "a __init__ constructor*"
        ]
    )

This creates a test file with a class having an `__init__` method, runs pytest, and asserts that the output contains a specific warning.

Mermaid Diagram: Class Structure

classDiagram
    class TestModule {
        +test_failing_import(pytester)
        +test_import_duplicate(pytester)
        +test_import_prepend_append(pytester, monkeypatch)
        +test_syntax_error_in_module(pytester)
        +test_module_considers_pluginmanager_at_import(pytester)
        +test_invalid_test_module_name(pytester)
        +test_show_traceback_import_error(pytester, verbose)
        +test_show_traceback_import_error_unicode(pytester)
    }
    class TestClass {
        +test_class_with_init_warning(pytester)
        +test_class_with_new_warning(pytester)
        +test_class_subclassobject(pytester)
        +test_static_method(pytester)
        +test_setup_teardown_class_as_classmethod(pytester)
        +test_issue1035_obj_has_getattr(pytester)
        +test_issue1579_namedtuple(pytester)
        +test_issue2234_property(pytester)
        +test_does_not_discover_properties(pytester)
        +test_does_not_discover_instance_descriptors(pytester)
        +test_abstract_class_is_not_collected(pytester)
    }
    class TestFunction {
        +test_getmodulecollector(pytester)
        +test_function_as_object_instance_ignored(pytester)
        +make_function(pytester, **kwargs)
        +test_function_equality(pytester)
        +test_repr_produces_actual_test_id(pytester)
        +test_issue197_parametrize_emptyset(pytester)
        +test_single_tuple_unwraps_values(pytester)
        +test_issue213_parametrize_value_no_equal(pytester)
        +test_parametrize_with_non_hashable_values(pytester)
        +test_parametrize_with_non_hashable_values_indirect(pytester)
        +test_parametrize_overrides_fixture(pytester)
        +test_parametrize_overrides_parametrized_fixture(pytester)
        +test_parametrize_overrides_indirect_dependency_fixture(pytester)
        +test_parametrize_with_mark(pytester)
        +test_parametrize_with_empty_string_arguments(pytester)
        +test_function_equality_with_callspec(pytester)
        +test_pyfunc_call(pytester)
        +test_multiple_parametrize(pytester)
        +test_issue751_multiple_parametrize_with_ids(pytester)
        +test_parametrize_skipif(pytester)
        +test_parametrize_skip(pytester)
        +test_parametrize_skipif_no_skip(pytester)
        +test_parametrize_xfail(pytester)
        +test_parametrize_passed(pytester)
        +test_parametrize_xfail_passed(pytester)
        +test_function_originalname(pytester)
        +test_function_with_square_brackets(pytester)
    }
    class TestSorting {
        +test_check_equality(pytester)
        +test_allow_sane_sorting_for_decorators(pytester)
        +test_ordered_by_definition_order(pytester)
    }
    class TestConftestCustomization {
        +test_pytest_pycollect_module(pytester)
        +test_customized_pymakemodule_issue205_subdir(pytester)
        +test_customized_pymakeitem(pytester)
        +test_pytest_pycollect_makeitem(pytester)
        +test_issue2369_collect_module_fileext(pytester)
        +test_early_ignored_attributes(pytester)
    }
    class TestTracebackCutting {
        +test_skip_simple()
        +test_traceback_argsetup(pytester)
        +test_traceback_error_during_import(pytester)
        +test_traceback_filter_error_during_fixture_collection(pytester)
        +test_filter_traceback_generated_code()
        +test_filter_traceback_path_no_longer_valid(pytester)
    }
    class TestReportInfo {
        +test_itemreport_reportinfo(pytester)
        +test_func_reportinfo(pytester)
        +test_class_reportinfo(pytester)
        +test_reportinfo_with_nasty_getattr(pytester)
    }
    
    TestModule ..|> object
    TestClass ..|> object
    TestFunction ..|> object
    TestSorting ..|> object
    TestConftestCustomization ..|> object
    TestTracebackCutting ..|> object
    TestReportInfo ..|> object

Summary

[collect.py](/projects/286/67331) contains a rich set of tests targeting the core pytest test collection mechanism. It validates:

• Collection of test modules, classes, and functions, including handling of invalid test code and import failures.

• Parametrization and its impact on test item generation.

• The effect of pytest collection hooks that allow customization.

• Correct ordering, equality, and hashing of collected test items.

• Proper formatting and filtering of tracebacks during errors.

• Handling of edge cases such as abstract classes, properties, descriptors, callable class instances, and duplicate test names.

• Interaction with filesystem and import machinery to simulate real-world test discovery scenarios.

This file is crucial for maintaining the reliability and correctness of pytest's test discovery and collection subsystem, ensuring that end-users experience consistent and predictable test runs.

End of Documentation for collect.py