test_unittest.py

Overview

`test_unittest.py` is a comprehensive test suite designed to verify the integration and functionality of Python's built-in `unittest` framework within the pytest testing environment. It leverages `pytest`'s own testing utilities (`Pytester`, `MonkeyPatch`) to programmatically generate, run, and assert behaviors of various `unittest.TestCase` scenarios.

The file ensures:

• Correct discovery, execution, and reporting of unittest-style tests.

• Proper handling of unittest lifecycle methods such as setUp, tearDown, setUpClass, tearDownClass, setUpModule, and tearDownModule.

• Compatibility with pytest features like fixtures, skipping tests, expected failures, and cleanup functions.

• Integration with asynchronous test cases and third-party unittest extensions (e.g., twisted.trial.unittest, asynctest).

• Robust handling of edge cases and regression issues (e.g., exception handling, traceback pruning, skip behaviors).

• Correct behavior of pytest's PDB (debugger) integration in unittest tests.

• Validation of pytest's test result and reporting mechanisms for unittest tests.

This file is essential for ensuring that pytest's unittest support remains stable, compatible, and feature-complete.

Detailed Descriptions

Key Imports

• pytest: Main test framework used.

• Pytester: Pytest's testing utility for creating test files and running pytest programmatically.

• MonkeyPatch: Utility for monkeypatching objects during tests.

• ExitCode: Enum for pytest exit codes.

Functions

Each function is a test case validating a particular aspect of unittest integration.

test_simple_unittest(pytester: Pytester) -> None

• Purpose: Tests that simple unittest test methods passing and failing are correctly reported.

• Parameters:

  • pytester: Fixture for programmatically creating test files and running pytest.

• Returns: None

• Usage: Creates a unittest.TestCase subclass with one passing and one failing test; asserts correct outcomes.

• Implementation Highlights:

  • Uses pytester.makepyfile to create test source.

  • Runs tests inline via pytester.inline_run.

  • Checks reports for pass/fail status.

test_runTest_method(pytester: Pytester) -> None

• Purpose: Verifies that unittest test cases with runTest method are run and reported correctly.

• Details: Tests both cases with only runTest and with additional test methods.

• Usage: Ensures verbosity output contains expected test names and indicates passes.

test_isclasscheck_issue53(pytester: Pytester) -> None

• Purpose: Regression test for issue #53, ensuring non-class objects with getattr don't break test collection.

• Outcome: No tests collected.

test_setup(pytester: Pytester) -> None

• Purpose: Tests that both setUp and pytest-style setup_method are called, and that teardown_method errors are reported.

• Outcome: Passing test with failing teardown triggers failure report containing error message "42".

test_setUpModule(pytester: Pytester) -> None

• Purpose: Tests module-level setup and teardown functions are called appropriately.

• Outcome: Both tests see the module-level setup effect; tests pass.

test_new_instances(pytester: Pytester) -> None

• Purpose: Verifies that new instances of test case classes are created for each test method (state is not shared).

• Usage: One test sets an attribute, another asserts it doesn't exist.

test_function_item_obj_is_instance(pytester: Pytester) -> None

• Purpose: Ensures that pytest's item.obj for unittest tests is a bound method on the test case instance, matching the test class.

• Related Issue: #5390

test_teardown(pytester: Pytester) -> None

• Purpose: Confirms tearDown methods execute and modify shared state as expected.

test_teardown_issue1649(pytester: Pytester) -> None

• Purpose: Regression test for issue #1649 ensuring unittest TestCase objects get garbage collected after tests to avoid memory leaks.

test_unittest_skip_issue148(pytester: Pytester) -> None

• Purpose: Tests that classes decorated with @unittest.skip are correctly skipped during pytest runs.

test_method_and_teardown_failing_reporting(pytester: Pytester) -> None

• Purpose: Asserts that failures in both test method and tearDown are reported correctly, including multiple failure reports.

test_setup_failure_is_shown(pytester: Pytester) -> None

• Purpose: Confirms that exceptions raised in setUp are reported, and that tests are not executed if setUp fails.

test_setup_setUpClass(pytester: Pytester) -> None

• Purpose: Tests class-level setup and teardown (setUpClass, tearDownClass) and that class attributes are correctly managed.

test_fixtures_setup_setUpClass_issue8394(pytester: Pytester) -> None

• Purpose: Tests that running pytest --fixtures on unittest classes with setUpClass does not produce docstring warnings.

test_setup_class(pytester: Pytester) -> None

• Purpose: Tests pytest-style setup_class and teardown_class methods on unittest classes (non-@classmethod style).

test_testcase_adderrorandfailure_defers(pytester: Pytester, type: str) -> None

• Purpose: Tests that addError and addFailure methods on unittest TestCase properly defer exceptions when invoked during run.

test_testcase_custom_exception_info(pytester: Pytester, type: str) -> None

• Purpose: Tests robustness of unittest result handling when custom incompatible exception info objects are passed to addError or addFailure.

test_testcase_totally_incompatible_exception_info(pytester: Pytester) -> None

• Purpose: Tests handling of totally incompatible exception info added to unittest test results, and that an informative error message is generated.

test_module_level_pytestmark(pytester: Pytester) -> None

• Purpose: Tests that module-level pytest markers (e.g., pytestmark = pytest.mark.xfail) affect unittest test outcomes correctly.

TestTrialUnittest (class)

• Purpose: Tests integration with twisted.trial.unittest, a unittest-like framework from Twisted.

• Key methods:

  • setup_class: Skip if Twisted is not installed, configures warnings.

  • Tests for correct collection, skips, errors, xfails, and PDB integration with Twisted's unittest.

• Special notes: Handles twisted reactor timing and Deferred objects.

test_djangolike_testcase(pytester: Pytester) -> None

• Purpose: Tests a custom unittest subclass that overrides call to implement Django-like pre- and post-test hooks.

• Checks: That _pre_setup, setUp, test, tearDown, and _post_teardown are called in order.

test_unittest_not_shown_in_traceback(pytester: Pytester) -> None

• Purpose: Ensures unittest assertions do not show internal unittest function names in tracebacks.

test_unorderable_types(pytester: Pytester) -> None

• Purpose: Regression test to ensure no TypeError is raised during test discovery on classes with custom __name__.

test_unittest_typerror_traceback(pytester: Pytester) -> None

• Purpose: Tests that TypeErrors in test method signature (unexpected parameters) are reported.

test_unittest_expected_failure_for_failing_test_is_xfail(pytester: Pytester, runner) -> None

• Purpose: Tests that unittest @expectedFailure on failing tests show as xfail in pytest and as expected failures in unittest CLI.

test_unittest_expected_failure_for_passing_test_is_fail(pytester: Pytester, runner) -> None

• Purpose: Tests that unittest @expectedFailure on passing tests show as unexpected success failure.

test_unittest_setup_interaction(pytester: Pytester, stmt: str) -> None

• Purpose: Tests interaction of pytest fixtures with unittest tests, including class and function scoped fixtures that return early (return or yield).

test_non_unittest_no_setupclass_support(pytester: Pytester) -> None

• Purpose: Tests that non-unittest test classes with setUpClass do not support class-level setup/teardown.

test_no_teardown_if_setupclass_failed(pytester: Pytester) -> None

• Purpose: Tests that tearDownClass is not called if setUpClass fails.

test_cleanup_functions(pytester: Pytester) -> None

• Purpose: Tests that cleanup functions registered with addCleanup are always called after each test, including failing tests.

test_issue333_result_clearing(pytester: Pytester) -> None

• Purpose: Regression test ensuring pytest internal result clearing does not prevent failure reporting.

test_unittest_raise_skip_issue748(pytester: Pytester) -> None

• Purpose: Tests that raising unittest.SkipTest during a test results in correct skipping behavior.

test_unittest_skip_issue1169(pytester: Pytester) -> None

• Purpose: Tests that unittest @skip decorator works correctly with pytest.

test_class_method_containing_test_issue1558(pytester: Pytester) -> None

• Purpose: Tests that methods marked with test = False are not collected as tests.

test_usefixtures_marker_on_unittest(base, pytester: Pytester) -> None

• Purpose: Tests that pytest's @pytest.mark.usefixtures works correctly on unittest.TestCase subclasses or plain objects.

test_testcase_handles_init_exceptions(pytester: Pytester) -> None

• Purpose: Regression test for exceptions raised in TestCase.init bubbling up correctly.

test_error_message_with_parametrized_fixtures(pytester: Pytester) -> None

• Purpose: Tests error messages when parametrized fixtures are incorrectly used in unittest test methods.

test_setup_inheritance_skipping(pytester: Pytester, test_name, expected_outcome) -> None

• Purpose: Tests pytest skipping behavior when setup methods skip tests at various levels (test, class, module).

test_BdbQuit(pytester: Pytester) -> None

• Purpose: Tests that raising bdb.BdbQuit in unittest tests is handled gracefully.

test_exit_outcome(pytester: Pytester) -> None

• Purpose: Tests that calling pytest.exit() in a unittest test stops test execution and reports correctly.

test_trace(pytester: Pytester, monkeypatch: MonkeyPatch) -> None

• Purpose: Tests pytest's --trace option integration with unittest tests using monkeypatching.

test_pdb_teardown_called(pytester: Pytester, monkeypatch: MonkeyPatch) -> None

• Purpose: Ensures tearDown methods are always called when pytest is run with --pdb.

test_pdb_teardown_skipped_for_functions(pytester: Pytester, monkeypatch: MonkeyPatch, mark: str) -> None

• Purpose: Ensures setUp and tearDown are not called for skipped tests when --pdb is enabled.

test_pdb_teardown_skipped_for_classes(pytester: Pytester, monkeypatch: MonkeyPatch, mark: str) -> None

• Purpose: Similar to above, but for skipped test classes.

test_async_support(pytester: Pytester) -> None

• Purpose: Tests unittest asyncio support via the unittest.async_case module.

test_asynctest_support(pytester: Pytester) -> None

• Purpose: Tests support for the third-party asynctest library.

test_plain_unittest_does_not_support_async(pytester: Pytester) -> None

• Purpose: Confirms that plain unittest does not support async test methods by default, leading to warnings.

Cleanup Tests (test_do_class_cleanups_on_success, test_do_class_cleanups_on_setupclass_failure, test_do_class_cleanups_on_teardownclass_failure, test_do_cleanups_on_success, test_do_cleanups_on_setup_failure, test_do_cleanups_on_teardown_failure)

• Purpose: Verify that cleanup functions registered with unittest's addCleanup and addClassCleanup are called exactly once and even if setup/teardown fails.

TestClassCleanupErrors (class)

• Purpose: Tests that exceptions raised within class cleanup functions are reported properly with sub-exceptions details.

test_traceback_pruning(pytester: Pytester) -> None

• Purpose: Regression test for traceback pruning on unittest failures.

test_raising_unittest_skiptest_during_collection(pytester: Pytester) -> None

• Purpose: Tests handling of unittest.SkipTest raised at module level during collection.

test_abstract_testcase_is_not_collected(pytester: Pytester) -> None

• Purpose: Tests that abstract unittest test cases (using abc.ABC) are not collected as tests.

Important Implementation Details

• Dynamic Test Creation: Uses pytester.makepyfile() to dynamically create test source files and run them.

• Inline Test Execution: pytester.inline_run() executes tests programmatically and returns reports for assertions.

• Parametrization: Uses pytest.mark.parametrize to test multiple variants of behaviors.

• Monkeypatching: Used to modify behavior during tests (e.g., exception info, PDB internals).

• Regression Coverage: Includes tests for previously reported bugs and issues.

• Third-Party Integration: Special tests validate integration with Twisted's trial and asynctest.

• Pytest Fixtures: Checks compatibility of pytest fixtures with unittest classes.

• Exception Handling: Confirms that exceptions in setup, teardown, and test bodies are reported correctly.

• PDB Integration: Tests ensure that pytest's debugger integration respects unittest lifecycle methods.

Interaction with System Components

• Pytest Core: This file exercises pytest's unittest plugin and test collection/execution mechanisms.

• Pytester Fixture: Core utility for testing pytest plugins and features by creating and running test files.

• Monkeypatching Utilities: Used to patch internal pytest and unittest behaviors during tests.

• Twisted Trial: Optional integration tested if Twisted is present.

• Asyncio and Asynctest: Optional async unittest support tested via imports.

• Pytest CLI: Many tests execute pytest with various CLI options (-v, -s, --trace, --pdb, etc.) to verify output and behavior.

Usage Examples

def test_simple_unittest_example(pytester):
    # Create a unittest file with one passing and one failing test
    test_file = pytester.makepyfile("""
        import unittest
        class TestExample(unittest.TestCase):
            def test_pass(self):
                self.assertEqual(1, 1)
            def test_fail(self):
                self.assertEqual(1, 0)
    """)
    # Run tests inline
    reports = pytester.inline_run(test_file)
    # Assert pass/fail outcomes
    assert reports.matchreport("test_pass").passed
    assert reports.matchreport("test_fail").failed

Visual Diagram: Class and Function Structure

classDiagram
    class TestTrialUnittest {
        +setup_class(cls)
        +test_trial_testcase_runtest_not_collected(pytester)
        +test_trial_exceptions_with_skips(pytester)
        +test_trial_error(pytester)
        +test_trial_pdb(pytester)
        +test_trial_testcase_skip_property(pytester)
        +test_trial_testfunction_skip_property(pytester)
        +test_trial_testcase_todo_property(pytester)
        +test_trial_testfunction_todo_property(pytester)
    }

    class TestClassCleanupErrors {
        +test_class_cleanups_failure_in_setup(pytester)
        +test_class_cleanups_failure_in_teardown(pytester)
        +test_class_cleanup_1_failure_in_teardown(pytester)
    }

    %% Functions outside classes
    class Functions {
        +test_simple_unittest(pytester)
        +test_runTest_method(pytester)
        +test_isclasscheck_issue53(pytester)
        +test_setup(pytester)
        +test_setUpModule(pytester)
        +test_new_instances(pytester)
        +test_function_item_obj_is_instance(pytester)
        +test_teardown(pytester)
        +test_teardown_issue1649(pytester)
        +test_unittest_skip_issue148(pytester)
        +test_method_and_teardown_failing_reporting(pytester)
        +test_setup_failure_is_shown(pytester)
        +test_setup_setUpClass(pytester)
        +test_fixtures_setup_setUpClass_issue8394(pytester)
        +test_setup_class(pytester)
        +test_testcase_adderrorandfailure_defers(pytester, type)
        +test_testcase_custom_exception_info(pytester, type)
        +test_testcase_totally_incompatible_exception_info(pytester)
        +test_module_level_pytestmark(pytester)
        +test_djangolike_testcase(pytester)
        +test_unittest_not_shown_in_traceback(pytester)
        +test_unorderable_types(pytester)
        +test_unittest_typerror_traceback(pytester)
        +test_unittest_expected_failure_for_failing_test_is_xfail(pytester, runner)
        +test_unittest_expected_failure_for_passing_test_is_fail(pytester, runner)
        +test_unittest_setup_interaction(pytester, stmt)
        +test_non_unittest_no_setupclass_support(pytester)
        +test_no_teardown_if_setupclass_failed(pytester)
        +test_cleanup_functions(pytester)
        +test_issue333_result_clearing(pytester)
        +test_unittest_raise_skip_issue748(pytester)
        +test_unittest_skip_issue1169(pytester)
        +test_class_method_containing_test_issue1558(pytester)
        +test_usefixtures_marker_on_unittest(base, pytester)
        +test_testcase_handles_init_exceptions(pytester)
        +test_error_message_with_parametrized_fixtures(pytester)
        +test_setup_inheritance_skipping(pytester, test_name, expected_outcome)
        +