pythoncollection.rst

Overview

This file serves as a comprehensive guide and reference documentation on how to customize and control Python test discovery and collection behavior when using the **pytest** testing framework. It explains various command-line options, configuration settings, and Python hooks that influence which tests pytest collects and runs.

The document covers topics such as ignoring files or directories during test collection, deselecting specific tests, handling duplicate test paths, modifying recursive directory traversal, changing naming conventions for test files and functions, interpreting command-line arguments as Python packages, and customizing test collection logic via `conftest.py`.

This is not source code but a rich documentation file (reStructuredText) designed to instruct users and developers on pytest’s test discovery customization capabilities through examples and detailed explanations.

Detailed Explanations of Key Topics

Ignoring Paths During Test Collection

• Purpose: Exclude certain test files or directories from collection.

• How: Use the CLI option --ignore=path to specify files or directories to skip.

• Multiple ignores: Allowed by repeating --ignore multiple times.

• Example usage:

  pytest --ignore=tests/foobar/test_foobar_03.py --ignore=tests/hello/
  

• Effect: pytest will collect tests only from files/directories not ignored.

• Wildcard support: Use --ignore-glob='pattern' to ignore files matching Unix shell-style wildcards.

Deselecting Individual Tests

• Purpose: Skip specific test functions or methods during collection.

• How: Use CLI option --deselect=item where item is the fully qualified test name.

• Multiple deselects: Allowed by repeating --deselect.

• Example usage:

  pytest --deselect tests/foobar/test_foobar_01.py::test_a
  

• Effect: All tests except test_a in the specified file will run.

Handling Duplicate Paths

• Default behavior: pytest ignores duplicate paths specified on the CLI.

• To collect duplicates: Use --keep-duplicates.

• Example:

  pytest --keep-duplicates path_a path_a
  

• Note: Specifying the same test file twice always collects it twice, regardless of --keep-duplicates.

Changing Directory Recursion

• Purpose: Control which directories pytest recurses into during test discovery.

• Configuration: Set norecursedirs in pytest.ini.

• Example:

  [pytest]
  norecursedirs = .svn _build tmp*
  

• Effect: Directories like .svn, _build, and any starting with tmp are skipped.

Changing Naming Conventions for Tests

• Purpose: Customize how pytest identifies test files, classes, and functions.

• Configurable options: python_files, python_classes, python_functions.

• Example: Use check_* prefix instead of test_:

  [pytest]
  python_files = check_*.py
  python_classes = Check
  python_functions = *_check
  

• Effect: pytest collects tests matching these patterns instead of defaults.

• Multiple patterns: Separate with spaces, e.g., python_files = test_*.py example_*.py.

• Note: These options don’t affect unittest.TestCase discovery.

Interpreting Command-Line Arguments as Python Packages

• Purpose: Run tests by specifying Python package names rather than file paths.

• How: Use --pyargs option.

• Example:

  pytest --pyargs unittest2.test.test_skipping -q
  

• Permanent setting: Add addopts = --pyargs in pytest.ini.

Viewing Collected Tests Without Running

• Purpose: Inspect which tests pytest will collect.

• How: Use --collect-only option.

• Example:

  pytest --collect-only pythoncollection.py
  

• Effect: Prints a tree of discovered tests without executing them.

Customizing Test Collection via conftest.py

• Purpose: Dynamically ignore files or patterns based on runtime conditions like Python version.

• How: Define collect_ignore and/or collect_ignore_glob lists in conftest.py.

• Example:

  import sys
  
  collect_ignore = ["setup.py"]
  if sys.version_info[0] > 2:
      collect_ignore.append("pkg/module_py2.py")
      collect_ignore_glob = ["*_py2.py"]
  

• Effect: Files like setup.py and Python 2-only files are excluded from collection.

• Disabling Test Collection on Classes: Set class attribute __test__ = False to prevent pytest from collecting that class.

• Handling Abstract Test Classes: Use a mixin to automatically set __test__ attribute, avoiding manual overrides.

Implementation Details & Algorithms

• Ignoring and deselecting tests is implemented via command-line options that filter the test collection tree.

• Duplicate path handling determines whether pytest merges or duplicates test items based on CLI flags.

• Directory recursion relies on patterns specified in norecursedirs to prune traversal.

• Naming convention changes adjust the glob patterns and regexes pytest uses to identify test files, classes, and functions.

• Dynamic collection ignores via conftest.py hook variables integrate Python logic to modify discovery behavior at runtime.

• The __test__ attribute is a pytest convention checked during collection to determine if a class should be treated as a test container.

• Package-based test discovery (--pyargs) uses Python’s import system to resolve module/package paths before running tests.

Interaction with Other System Parts

• This file documents features directly related to pytest’s test collection subsystem, which is a core part of the pytest testing framework.

• It influences how pytest’s test discovery engine traverses directories, matches test files, and loads test functions and classes.

• It interacts with pytest configuration files (pytest.ini, tox.ini, setup.cfg) and hook files (conftest.py) to customize test collection behavior.

• It impacts how test runners and CI systems invoke pytest to include or exclude tests dynamically.

• It also relates to Python’s import system when resolving packages with --pyargs.

Usage Examples Summary

1. Ignoring a directory and a file:

   pytest --ignore=tests/foobar/test_foobar_03.py --ignore=tests/hello/
   

2. Deselecting a single test function:

   pytest --deselect tests/foobar/test_foobar_01.py::test_a
   

3. Collecting duplicate items:

   pytest --keep-duplicates path_a path_a
   

4. Skipping recursion in certain directories:

   [pytest]
   norecursedirs = .svn _build tmp*
   

5. Changing test naming conventions:

   [pytest]
   python_files = check_*.py
   python_classes = Check
   python_functions = *_check
   

6. Ignoring files dynamically in conftest.py:

   import sys
   collect_ignore = ["setup.py"]
   if sys.version_info[0] > 2:
       collect_ignore.append("pkg/module_py2.py")
   

Visual Diagram: Test Collection Customization Structure

flowchart TD
    A[pytest Test Collection] --> B[Command Line Options]
    A --> C[Configuration Files (pytest.ini)]
    A --> D[conftest.py Hooks]
    B --> B1(--ignore)
    B --> B2(--ignore-glob)
    B --> B3(--deselect)
    B --> B4(--keep-duplicates)
    B --> B5(--pyargs)
    C --> C1(python_files)
    C --> C2(python_classes)
    C --> C3(python_functions)
    C --> C4(norecursedirs)
    C --> C5(addopts)
    D --> D1(collect_ignore)
    D --> D2(collect_ignore_glob)
    D --> D3(__test__ attribute)
    A --> E[Test Discovery Process]
    E --> F[Directory Traversal]
    E --> G[File & Class Matching]
    E --> H[Test Item Collection]

• Explanation:
  The diagram illustrates how pytest’s test collection process is influenced by three main customization sources: command-line options, configuration files, and conftest.py hooks. These inputs affect directory traversal, file/class matching, and final test item collection.

Summary

This documentation file is a detailed manual on customizing pytest’s Python test discovery mechanisms. It provides users with practical options and techniques to control which tests are collected and run, enhancing test suite flexibility and reliability. Understanding and applying these settings can optimize test runs, avoid unwanted tests, and conform pytest’s behavior to project-specific needs.

**End of Documentation**