test_terminal.py

Overview

`test_terminal.py` is a comprehensive test suite designed to validate the terminal reporting functionality of the `pytest` testing framework. It focuses on how test results, collection statuses, warnings, errors, and other test session events are reported in the terminal output. The file ensures that the terminal reporter behaves correctly under various conditions, including different verbosity levels, output styles, test outcome scenarios (pass, fail, skip, xfail, xpass), and integrations with plugins such as [xdist](/projects/286/67223).

The tests cover both standard and edge cases, including:

• Basic test run reporting (pass, skip, fail).

• Collection-only mode output.

• Fixture setup and teardown error reporting.

• KeyboardInterrupt handling.

• Output styles like classic, progress, times, and count.

• Color output and theme handling.

• Summary and warning reporting.

• Handling of test deselection and reruns.

• Special cases like long skip reasons, internal errors, and plugin name/version formatting.

• Unicode and formatting edge cases in terminal output.

This file primarily interacts with the [TerminalReporter](/projects/286/67388) class and related utilities from [_pytest.terminal](/projects/286/67388), as well as pytest core APIs and plugins, to verify their output behavior through `pytester` test runs.

Detailed Documentation

Classes

1. DistInfo (NamedTuple)

• Purpose: Represents distribution information for plugins with project name and version.

• Attributes:

  • project_name: str — Plugin or project name.

  • version: int — Version number.

• Usage: Used in tests that verify plugin name/version formatting and deduplication.

2. Option

• Purpose: Helper class to encapsulate verbosity options for test runs.

• Attributes:

  • verbosity: int — Verbosity level (default 0).

• Properties:

  • args (list[str]): Returns a list of CLI arguments representing the verbosity level, e.g., ["--verbosity=1"].

• Usage: Used as a parameter in tests to run pytest with different verbosity settings.

3. TestTerminal

• Purpose: Tests core terminal reporting for test results with various verbosity levels and test outcomes.

• Key Methods:

  • test_pass_skip_fail(pytester: Pytester, option: Option): Validates terminal output for passing, skipping, and failing tests.

  • test_console_output_style_times_with_skipped_and_passed(pytester: Pytester): Ensures output style "times" shows skipped and passed counts.

  • test_internalerror(pytester: Pytester, linecomp): Tests reporting of internal errors.

  • test_writeline(pytester: Pytester, linecomp): Tests writing lines and file path results.

  • test_show_runtest_logstart(pytester: Pytester, linecomp): Tests display of test start logs.

  • test_runtest_location_shown_before_test_starts(pytester: Pytester): Ensures location is shown before test execution.

  • Various others testing keyboard interrupts, summary, color output, and more.

4. TestCollectonly

• Purpose: Tests the collection-only mode (--collect-only) terminal output.

• Key Methods:

  • test_collectonly_basic(pytester: Pytester): Basic collection output test.

  • test_collectonly_skipped_module(pytester: Pytester): Handles skipped module during collection.

  • test_collectonly_displays_test_description(pytester: Pytester, dummy_yaml_custom_test): Ensures test descriptions are shown.

  • test_collectonly_failed_module(pytester: Pytester): Handles failures during collection.

  • test_collectonly_fatal(pytester: Pytester): Tests collection fatal errors.

  • Other tests for quiet mode, missing paths, and error states.

5. TestFixtureReporting

• Purpose: Tests reporting of setup and teardown fixture errors in terminal output.

• Key Methods:

  • test_setup_fixture_error(pytester: Pytester): Setup error reporting.

  • test_teardown_fixture_error(pytester: Pytester): Teardown error reporting.

  • test_teardown_fixture_error_and_test_failure(pytester: Pytester): Combined test failure and teardown error reporting.

  • test_setup_teardown_output_and_test_failure(pytester: Pytester): Output capturing for setup/teardown with test failure.

6. TestTerminalFunctional

• Purpose: Functional tests for terminal reporting features like deselected tests, headers, summaries, verbosity, and output styles.

• Key Methods:

  • test_deselected(pytester: Pytester): Tests reporting of deselected tests.

  • test_verbose_reporting(verbose_testfile, pytester: Pytester): Tests verbose mode outputs.

  • test_quiet_reporting(pytester: Pytester): Tests quiet mode reporting.

  • test_header_trailer_info(monkeypatch: MonkeyPatch, pytester: Pytester, request): Tests session start headers and plugin listing.

  • test_showlocals(pytester: Pytester): Tests local variables display on failures.

  • test_color_yes, test_color_no: Tests colored output handling.

  • Other many tests covering various terminal reporting scenarios and command line options.

7. TestGenericReporting

• Purpose: Generic tests that can be adapted to different option providers, covering maxfailures, traceback options, collection failures.

• Key Methods:

  • test_collect_fail(pytester: Pytester, option): Tests collection failure reporting.

  • test_maxfailures(pytester: Pytester, option): Tests max failure stop behavior.

  • test_tb_option(pytester: Pytester, option): Tests different traceback modes.

  • test_tb_crashline(pytester: Pytester, option): Tests crash line traceback style.

  • test_pytest_report_header(pytester: Pytester, option): Custom header reporting.

8. TestClassicOutputStyle

• Purpose: Tests the "classic" console output style.

• Key Methods:

  • test_normal_verbosity(pytester: Pytester, test_files)

  • test_verbose(pytester: Pytester, test_files)

  • test_quiet(pytester: Pytester, test_files)

9. TestProgressOutputStyle

• Purpose: Tests the "progress" and related output styles.

• Key Methods:

  • test_normal(many_tests_files, pytester: Pytester)

  • test_colored_progress(pytester: Pytester, monkeypatch, color_mapping)

  • test_count(many_tests_files, pytester: Pytester)

  • test_times(many_tests_files, pytester: Pytester)

  • test_xdist_normal(many_tests_files, pytester: Pytester, monkeypatch)

  • Various others testing progress bar, color, verbosity, capture settings.

10. TestProgressWithTeardown

• Purpose: Ensures correct progress percentages with tests that fail during teardown.

• Key Methods:

  • test_teardown_simple(pytester: Pytester, contest_with_teardown_fixture)

  • test_teardown_with_test_also_failing(pytester: Pytester, contest_with_teardown_fixture)

  • test_teardown_many(pytester: Pytester, many_files)

  • test_xdist_normal(many_files, pytester: Pytester, monkeypatch)

11. TestFineGrainedTestCase

• Purpose: Tests fine-grained reporting per test case with various verbosity levels.

• Key Methods:

  • test_execute_positive(self, verbosity, pytester: Pytester)

  • test_execute_0_global_1(self, pytester: Pytester)

  • test_execute_negative(self, verbosity, pytester: Pytester)

  • test_execute_skipped_positive_2(self, pytester: Pytester)

  • test__collect_only_positive(self, verbosity, pytester: Pytester)

  • _initialize_files(pytester: Pytester, verbosity: int, file_contents: str) -> Path: Helper method to create test files with given verbosity.

Functions

Several standalone test functions exist to verify specific terminal reporter behaviors:

• test_plugin_nameversion(input, expected): Tests plugin name/version formatting and deduplication.

• test_fail_extra_reporting(pytester: Pytester, monkeypatch, use_ci: bool, expected_message: str): Tests failure message truncation depending on CI environment.

• test_fail_reporting_on_pass(pytester: Pytester): Ensures failure reporting not shown on passing tests.

• test_pass_extra_reporting(pytester: Pytester): Tests pass reporting.

• test_pass_reporting_on_fail(pytester: Pytester): Ensures pass reporting not shown on failing tests.

• test_pass_output_reporting(pytester: Pytester): Tests output capturing on passing tests.

• test_color_yes(pytester: Pytester, color_mapping): Tests colored output with --color=yes.

• test_color_no(pytester: Pytester): Tests output without color.

• test_color_yes_collection_on_non_atty(pytester: Pytester, verbose): Tests color output respects terminal type.

• test_getreportopt(): Tests report option parsing logic.

• test_terminalreporter_reportopt_addopts(pytester: Pytester): Tests that addopts are respected in terminalreporter.

• test_tbstyle_short(pytester: Pytester): Tests short traceback style.

• test_traceconfig(pytester: Pytester): Tests config tracing output.

• test_skip_reasons_folding(): Tests folding behavior for skip reasons.

• test_line_with_reprcrash(monkeypatch: MonkeyPatch): Tests formatting of lines with reprcrash messages for terminal output.

• test_short_summary_with_verbose(monkeypatch: MonkeyPatch, pytester: Pytester): Tests truncation of summary info at different verbosity levels.

• test_full_sequence_print_with_vv(monkeypatch: MonkeyPatch, pytester: Pytester): Tests sequence printing in summary with -vv.

• test_force_short_summary(monkeypatch: MonkeyPatch, pytester: Pytester): Tests forced short summary truncation.

• test_format_session_duration(seconds, expected): Tests session duration formatting.

• test_format_node_duration(seconds, expected): Tests node duration formatting.

• test_collecterror(pytester: Pytester): Tests reporting of collection errors.

• test_no_summary_collecterror(pytester: Pytester): Tests disabling summary on collection errors.

• test_via_exec(pytester: Pytester): Tests test definition via exec.

• test_raw_skip_reason_skipped(): Tests extraction of skip reason from report.

• test_raw_skip_reason_xfail(): Tests extraction of xfail reason from report.

• test_format_trimmed(): Tests trimming of skip reason messages.

• test_summary_xfail_reason(pytester: Pytester): Tests xfail reason summary output.

• test_xfail_tb_default(xfail_testfile, pytester: Pytester): Tests default traceback for xfail tests.

• test_xfail_tb_true(xfail_testfile, pytester: Pytester): Tests showing traceback for xfail tests with flag.

• test_xfail_tb_line(xfail_testfile, pytester: Pytester): Tests line traceback for xfail tests.

• test_summary_xpass_reason(pytester: Pytester): Tests xpass reason summary output.

• test_xpass_output(pytester: Pytester): Tests output capturing for xpass tests.

• test_nodeid_handling_windows_paths(pytester: Pytester, tmp_path): Tests nodeid handling for Windows-style paths.

Important Implementation Details and Algorithms

• Plugin Name Version Formatting: Uses _plugin_nameversions function to normalize plugin names by stripping prefixes and deduplicating.

• Terminal Output Styles: Tests cover multiple console output styles (classic, progress, count, times) and their effect on how test results are displayed.

• Skip Reason Folding: Uses _folded_skips to fold multiple skip reports into a concise summary.

• Line Trimming and Formatting: _get_line_with_reprcrash_message is tested to ensure messages fit into terminal width, trimming with ellipses as necessary, while correctly handling unicode width.

• Colored Output: Tests verify correct application of color codes and theme selection, including validating environment variables PYTEST_THEME and PYTEST_THEME_MODE.

• Test Selection Reporting: Tests verify correct counts and reporting when tests are deselected, skipped, or filtered by markers or keywords.

• Traceback Styles: Various tests confirm that traceback formatting (long, short, line, no) behaves as expected.

• Fixture Setup/Teardown Errors: Tests ensure errors during setup or teardown are reported with captured output and correct formatting.

• Summary Line Color Logic: Tests verify the logic determining summary line color based on test results, with "boring" statuses (e.g., skipped) not affecting color.

• Handling of KeyboardInterrupt: Tests verify proper reporting and behavior when tests or sessions are interrupted by keyboard.

Interaction with Other Parts of the System

• This test file exercises the TerminalReporter class and utilities from the _pytest.terminal module to validate terminal output behavior.

• It interacts heavily with the pytester testing fixture, which provides a way to create temporary test files, run pytest, and analyze terminal output.

• Uses hooks like pytest_report_teststatus, pytest_terminal_summary, and others to verify integration points.

• Uses imports from _pytest.reports for creating fake reports to test summary and skip folding.

• Interacts with plugins such as xdist (distributed testing) to validate terminal reporting in parallel test execution scenarios.

• Validates environment variable influences on output (e.g., PY_COLORS, PYTEST_THEME).

Usage Examples

Most usage is through pytest's own testing framework (`pytester`) for running tests and capturing terminal output. Examples include:

def test_pass_skip_fail(self, pytester: Pytester, option: Option) -> None:
    pytester.makepyfile(
        '''
        import pytest
        def test_ok():
            pass
        def test_skip():
            pytest.skip("xx")
        def test_func():
            assert 0
        '''
    )
    result = pytester.runpytest(*option.args)
    # Assertions on terminal output depending on verbosity
    ...

Mermaid Class Diagram

classDiagram
    class DistInfo {
        +str project_name
        +int version
    }
    class Option {
        -int verbosity
        +__init__(verbosity=0)
        +args() list~str~
    }
    class TestTerminal {
        +test_pass_skip_fail(pytester, option)
        +test_console_output_style_times_with_skipped_and_passed(pytester)
        +test_internalerror(pytester, linecomp)
        +test_writeline(pytester, linecomp)
        +test_show_runtest_logstart(pytester, linecomp)
        +test_runtest_location_shown_before_test_starts(pytester)
        +test_report_collect_after_half_a_second(pytester, monkeypatch)
        +test_itemreport_subclasses_show_subclassed_file(pytester)
        +test_itemreport_directclasses_not_shown_as_subclasses(pytester)
        +test_keyboard_interrupt(pytester, fulltrace)
        +test_keyboard_in_sessionstart(pytester)
        +test_collect_single_item(pytester)
        +test_rewrite(pytester, monkeypatch)
        +test_report_teststatus_explicit_markup(monkeypatch, pytester, color_mapping, category)
        +test_verbose_skip_reason(pytester)
        +test_isatty(pytester, monkeypatch, isatty)
    }
    class TestCollectonly {
        +test_collectonly_basic(pytester)
        +test_collectonly_skipped_module(pytester)
        +test_collectonly_displays_test_description(pytester, dummy_yaml_custom_test)
        +test_collectonly_failed_module(pytester)
        +test_collectonly_fatal(pytester)
        +test_collectonly_simple(pytester)
        +test_collectonly_error(pytester)
        +test_collectonly_missing_path(pytester)
        +test_collectonly_quiet(pytester)
        +test_collectonly_more_quiet(pytester)
        +test_collect_only_summary_status(pytester)
    }
    class TestFixtureReporting {
        +test_setup_fixture_error(pytester)
        +test_teardown_fixture_error(pytester)
        +test_teardown_fixture_error_and_test_failure(pytester)
        +test_setup_teardown_output_and_test_failure(pytester)
    }
    class TestTerminalFunctional {
        +test_deselected(pytester)
        +test_deselected_with_hook_wrapper(pytester)
        +test_show_deselected_items_using_markexpr_before_test_execution(pytester)
        +test_selected_count_with_error(pytester)
        +test_no_skip_summary_if_failure(pytester)
        +test_passes(pytester)
        +test_header_trailer_info(monkeypatch, pytester, request)
        +test_no_header_trailer_info(monkeypatch, pytester, request)
        +test_header(pytester)
        +test_header_absolute_testpath(pytester, monkeypatch)
        +test_no_header(pytester)
        +test_no_summary(pytester)
        +test_showlocals(pytester)
        +test_noshowlocals_addopts_override(pytester)
        +test_showlocals_short(pytester)
        +test_verbose_reporting(verbose_testfile, pytester)
        +test_verbose_reporting_xdist(verbose_testfile, monkeypatch, pytester, pytestconfig)
        +test_quiet_reporting(pytester)
        +test_more_quiet_reporting(pytester)
        +test_report_collectionfinish_hook(pytester, params)
        +test_summary_f_alias(pytester)
        +test_summary_s_alias(pytester)
        +test_summary_s_folded(pytester)
        +test_summary_s_unfolded(pytester)
    }
    class TestGenericReporting {
        +test_collect_fail(pytester, option)
        +test_maxfailures(pytester, option)
        +test_maxfailures_with_interrupted(pytester)
        +test_tb_option(pytester, option)
        +test_tb_crashline(pytester, option)
        +test_tb_crashline_pytrace_false(pytester, option)
        +test_pytest_report_header(pytester, option)
        +test_show_capture(pytester)
        +test_show_capture_with_teardown_logs(pytester)
    }
    class TestClassicOutputStyle {
        +test_normal_verbosity(pytester, test_files)
        +test_verbose(pytester, test_files)
        +test_quiet(pytester, test_files)
    }
    class TestProgressOutputStyle {
        +test_zero_tests_collected(pytester)
        +test_normal(many_tests_files, pytester)
        +test_colored_progress(pytester