test_setuponly.py

Overview

`test_setuponly.py` is a test suite designed to validate and verify pytest's fixture setup reporting features, specifically for the pytest command-line options `--setup-only`, `--setup-plan`, and `--setup-show`. These options control how pytest reports the setup and teardown phases of test fixtures without necessarily running the test body (or running it partially).

This file contains multiple pytest test functions that create and run temporary test files with various fixture configurations to ensure that pytest correctly displays fixture setup/teardown actions, supports different fixture scopes, handles parameterized fixtures, autouse fixtures, dynamic fixture requests, output capturing, and even special cases like KeyboardInterrupt exceptions.

The tests leverage the internal pytest `Pytester` utility to programmatically create test modules and conftest files and run pytest subprocesses with the specified setup options, then assert on the output to confirm expected setup/teardown logs.

Detailed Explanation of Key Components

Imports

• pytest - The testing framework itself.

• Pytester from _pytest.pytester - Pytest's internal testing helper to create test files and run pytest in subprocesses.

• ExitCode from _pytest.config - Enum for pytest exit codes.

• sys - Used in one test to run a subprocess with warnings enabled.

Fixture: mode

@pytest.fixture(params=["--setup-only", "--setup-plan", "--setup-show"], scope="module")
def mode(request):
    return request.param

• Purpose: Parametrizes tests to run with each of the three setup modes.

• Parameters: None.

• Returns: One of the strings "--setup-only", "--setup-plan", or "--setup-show".

• Usage: Injected into test functions to run the same test logic under all three pytest setup reporting options.

Test Functions

All test functions use the `pytester` fixture to create temporary test files and run pytest subprocesses.

test_show_only_active_fixtures

• Purpose: Verify that only active fixtures (those used by tests) are shown and hidden fixtures are excluded.

• Parameters:

  • pytester: Pytester

  • mode: str - one of the setup modes.

  • dummy_yaml_custom_test - a fixture (not detailed here) presumably providing some baseline test environment.

• Behavior:

  • Creates a test file with two fixtures: a hidden fixture _arg0 and a visible fixture arg1.

  • Runs pytest with the current mode.

  • Asserts that setup/teardown output only references arg1.

• Example usage:

  pytest --setup-show
  

• Return: None (assertions validate behavior).

test_show_different_scopes

• Purpose: Test fixture setup/teardown output with fixtures of different scopes (function, session).

• Details:

  • Creates a function-scoped fixture and a session-scoped fixture.

  • Checks that their setup/teardown messages appear correctly with appropriate scope prefixes (F for function, S for session).

test_show_nested_fixtures

• Purpose: Validate nested fixture setups where a function-scoped fixture depends on a session-scoped fixture with the same name.

• Details:

  • Uses a conftest fixture with session scope arg_same.

  • Defines a function-scoped fixture arg_same using the session-scoped fixture.

  • Expects setup logs showing nested setup calls.

test_show_fixtures_with_autouse

• Purpose: Ensure autouse fixtures (automatically used without explicit test argument) are shown in setup output.

• Details:

  • Defines a session-scoped autouse fixture.

  • Verifies it appears in setup logs even if not explicitly requested by the test.

test_show_fixtures_with_parameters

• Purpose: Test fixtures parameterized with multiple values.

• Details:

  • Defines a session-scoped parameterized fixture with values foo and bar.

  • Checks that setup and teardown for each parameter variant appear in the output.

test_show_fixtures_with_parameter_ids

• Purpose: Test parameterized fixtures with custom string IDs.

• Details:

  • Uses ids=['spam', 'ham'] for parameters.

  • Verifies that these IDs are shown in the setup output.

test_show_fixtures_with_parameter_ids_function

• Purpose: Test parameterized fixtures with a function that generates parameter IDs.

• Details:

  • Uses an ID function that uppercases parameter values.

  • Checks that the uppercase IDs appear in the setup output.

test_dynamic_fixture_request

• Purpose: Validate that fixtures dynamically requested using request.getfixturevalue() are shown in setup output.

• Details:

  • Defines a fixture that calls request.getfixturevalue() inside its body.

  • Confirms that the dynamically requested fixture is set up and torn down properly.

test_capturing

• Purpose: Test that stdout and stderr output within fixtures is captured and displayed when running with setup-only options.

• Details:

  • Writes to stdout and stderr in a fixture.

  • Checks that the output appears in pytest's captured output.

test_show_fixtures_and_execute_test

• Purpose: Verify that with --setup-show, pytest both shows fixture setup/teardown and executes the test body.

• Details:

  • Creates a test that asserts False to ensure test failure.

  • Checks that setup and teardown logs appear and test failure is reported.

test_setup_show_with_KeyboardInterrupt_in_test

• Purpose: Test pytest behavior when a test raises a KeyboardInterrupt.

• Details:

  • Runs a test that raises KeyboardInterrupt.

  • Checks that setup and teardown logs appear, the interrupt message is shown, and exit code matches ExitCode.INTERRUPTED.

test_show_fixture_action_with_bytes

• Purpose: Regression test for issue #7126 regarding BytesWarning when using --setup-show with byte-string parameters.

• Details:

  • Parametrizes a test with a bytes literal.

  • Runs pytest with Python warnings enabled (-bb).

  • Checks that no warnings cause failure and the test passes.

Important Implementation Details

• Use of pytester
  The pytester fixture is central to this file. It allows dynamically creating test modules and running pytest commands in subprocesses, capturing output for assertions. This approach is essential for testing pytest's own CLI behavior and output formatting.

• Parametrization using mode fixture
  Tests are run three times each with different pytest CLI options for fixture setup reporting. This ensures consistent and comprehensive coverage of all related CLI modes.

• Output matching
  Tests use result.stdout.fnmatch_lines() (which supports glob-style wildcards) to verify that specific lines appear in the output, validating the presence and formatting of fixture setup/teardown logs.

• Handling of special cases
  Tests cover autouse fixtures, dynamically requested fixtures, parameterized fixtures with IDs (including functional IDs), capturing of output, and even interrupt handling, demonstrating thorough test coverage of corner cases.

Interaction with Other Parts of the System

• pytest Core
  This test file exercises pytest's fixture setup and reporting internals. It interacts with pytest's fixture system, collection, and terminal reporting subsystems.

• Conftest and Test Modules
  Tests dynamically create temporary conftest files and test modules to simulate different fixture scenarios.

• Exit Codes
  Uses pytest's ExitCode enum to verify proper exit statuses for interrupted runs.

• Standard Output
  Checks the output printed to the console by pytest during setup and teardown phases.

Usage Example

To manually reproduce some tests in this file, you could run pytest with:

pytest --setup-show

or

pytest --setup-only

on your test suite to see detailed fixture setup and teardown information printed to the console.

Mermaid Class Diagram

This file mainly contains test functions; no classes are declared. However, the key entity is the parametrized fixture `mode` and the multiple test functions that use `pytester` and `mode`. Representing this as a flowchart of test execution and fixture injection is more appropriate.

flowchart TD
    A[mode fixture] -->|parametrize| B[test_show_only_active_fixtures]
    A --> C[test_show_different_scopes]
    A --> D[test_show_nested_fixtures]
    A --> E[test_show_fixtures_with_autouse]
    A --> F[test_show_fixtures_with_parameters]
    A --> G[test_show_fixtures_with_parameter_ids]
    A --> H[test_show_fixtures_with_parameter_ids_function]
    B --> I[pytester fixture]
    C --> I
    D --> I
    E --> I
    F --> I
    G --> I
    H --> I
    I --> J[Creates test files and runs pytest subprocess]
    J --> K[Captures output and return code]
    K --> L[Assertions on setup/teardown output]

Summary

`test_setuponly.py` is a comprehensive pytest test module that validates the correctness of pytest's fixture setup reporting CLI options. It tests fixture visibility, scope handling, parameterization, autouse behavior, dynamic fixture requests, output capturing, and error handling. Leveraging pytest's own internal testing utilities, it ensures pytest's user-facing fixture setup output remains accurate and informative across a variety of scenarios.

This file is a critical part of pytest's own test suite, ensuring high quality and reliability of fixture reporting features that aid users in debugging and understanding fixture usage during test runs.