failure_demo.py

Overview

`failure_demo.py` is a Python test suite primarily designed to demonstrate various failure cases and assertion error scenarios using the `pytest` testing framework. The file contains multiple test classes and functions that intentionally fail or raise exceptions to illustrate how `pytest` reports failures, compares data structures, and handles error conditions.

This file serves as a comprehensive showcase of different assertion patterns, error types, and test parametrizations, which can be valuable for:

• Testing the robustness of pytest’s failure reporting and introspection capabilities.

• Demonstrating detailed pytest failure messages for different data types (strings, lists, dictionaries, sets).

• Illustrating how pytest handles exceptions, unpacking errors, attribute access failures, and dynamic code execution.

• Providing examples of custom assert messages and specialized explanations for test failures.

Detailed Breakdown

1. Functions

otherfunc(a, b)

• Purpose: Asserts that a equals b.

• Parameters:

  • a: Any type, expected to be comparable.

  • b: Any type, expected to be comparable.

• Returns: None.

• Usage: Called by somefunc and other tests to trigger assertion failures.

• Example:

  otherfunc(5, 5)  # Passes
  otherfunc(5, 6)  # Raises AssertionError
  

somefunc(x, y)

• Purpose: Calls otherfunc(x, y).

• Parameters:

  • x, y: Any types.

• Returns: None.

• Usage: Indirect assertion via otherfunc.

• Example:

  somefunc(10, 10)  # Passes
  somefunc(10, 20)  # Raises AssertionError
  

otherfunc_multi(a, b)

• Purpose: Same as otherfunc, asserts equality.

• Parameters:

  • a, b: Any comparable types.

• Returns: None.

globf(x)

• Purpose: Returns x + 1.

• Parameters:

  • x: Numeric (int, float).

• Returns: Numeric, one greater than input.

• Example:

  globf(5)  # returns 6
  

2. Test Functions (Outside Classes)

test_generative(param1, param2)

• Purpose: Parametrized test with one tuple (3, 6).

• Asserts: param1 * 2 < param2.

• Expected to Pass: 3 * 2 = 6, which is NOT less than 6, so this will fail.

• Usage: Demonstrates pytest parameterization and failure on comparison.

test_attribute(), test_attribute_instance(), test_attribute_failure(), test_attribute_multiple()

• Purpose: Test attribute access on classes and instances.

• Highlights: Showcases failures when expected attribute values do not match or raise exceptions.

test_dynamic_compile_shows_nicely()

• Purpose: Dynamically compiles and executes a function that asserts 1 == 0.

• Demonstrates: How pytest handles dynamically compiled code and failure reporting.

3. Test Classes

class TestFailing

• Contains tests that intentionally fail simple assertions.

• Methods:

  • test_simple(self): Defines functions f (returns 42) and g (returns 43), asserts f() == g(), which fails.

  • test_simple_multiline(self): Calls otherfunc_multi(42, 54) (since 6*9=54), which asserts equality and fails.

  • test_not(self): Defines f returning 42, asserts not f() (i.e., not 42), which fails because 42 is truthy.

class TestSpecialisedExplanations

• Contains many tests comparing strings, lists, dicts, sets, and dataclasses with subtle differences to demonstrate pytest’s detailed failure explanations.

• Examples include:

  • Comparing similar but unequal strings.

  • Long multiline string comparisons.

  • Collection comparisons with different elements or missing keys.

  • in and not in assertion failures on multiline or long strings.

  • Dataclass and attr-based class equality failures.

class TestRaises

• Tests exceptions and error conditions.

• Uses pytest.raises to check for exceptions.

• Includes tests that raise exceptions deliberately to demonstrate failure cases.

• Examples:

  • test_raises: expects int("qwe") to raise TypeError.

  • test_raises_doesnt: expects int("3") to raise OSError (which it does not, so fails).

  • test_raise: raises ValueError directly.

  • test_tupleerror: tries unpacking a list with insufficient elements.

  • test_reinterpret_fails_with_print_for_the_fun_of_it: shows failure after printing debug info.

  • test_some_error: references undefined variable namenotexi.

  • func1: asserts false (41 == 42).

class TestMoreErrors

• Tests various assertion failure scenarios, including:

  • Assertion with functions returning different values.

  • Unpacking errors with empty or invalid iterables.

  • String method assertion failures (startswith).

  • Type mismatch assertions.

  • Attribute error scenarios (self.x usage).

  • Comparison failures.

  • Assertion within try-finally blocks.

class TestCustomAssertMsg

• Demonstrates usage of custom assertion messages.

• Tests with single line and multiline messages.

• Shows custom object representations in assertion messages.

Important Implementation Details

• The file uses plain assert statements extensively to trigger assertion failures.

• Uses pytest.mark.parametrize for parameterized testing.

• Uses pytest.raises as a context manager and function to check expected exceptions.

• Contains nested function definitions within test methods to generate local scopes for testing.

• Uses dataclasses and attr to test equality of complex objects.

• Includes tests that explore how pytest formats failure messages for complex data structures and dynamic code.

Interaction With Other Parts of the System

• This file is intended to be run by pytest in a testing environment.

• It does not import or depend on any other project-specific modules.

• It imports standard Python modules (dataclasses, attr, importlib.util, sys) only in local scopes inside test methods.

• It interacts with pytest’s core assertion rewriting and failure reporting mechanisms by deliberately triggering diverse failures.

• The file can be used as part of a larger test suite to validate pytest's reporting or as an educational tool for writing tests.

Usage Examples

To run the tests and see the failure reports:

pytest failure_demo.py

You will observe multiple assertion failures, unpacking errors, exception mismatches, and detailed comparison diffs generated by pytest.

Visual Diagram

classDiagram
    class TestFailing {
        +test_simple()
        +test_simple_multiline()
        +test_not()
    }

    class TestSpecialisedExplanations {
        +test_eq_text()
        +test_eq_similar_text()
        +test_eq_multiline_text()
        +test_eq_long_text()
        +test_eq_long_text_multiline()
        +test_eq_list()
        +test_eq_list_long()
        +test_eq_dict()
        +test_eq_set()
        +test_eq_longer_list()
        +test_in_list()
        +test_not_in_text_multiline()
        +test_not_in_text_single()
        +test_not_in_text_single_long()
        +test_not_in_text_single_long_term()
        +test_eq_dataclass()
        +test_eq_attrs()
    }

    class TestRaises {
        +test_raises()
        +test_raises_doesnt()
        +test_raise()
        +test_tupleerror()
        +test_reinterpret_fails_with_print_for_the_fun_of_it()
        +test_some_error()
        +func1()
    }

    class TestMoreErrors {
        +test_complex_error()
        +test_z1_unpack_error()
        +test_z2_type_error()
        +test_startswith()
        +test_startswith_nested()
        +test_global_func()
        +test_instance()
        +test_compare()
        +test_try_finally()
    }

    class TestCustomAssertMsg {
        +test_single_line()
        +test_multiline()
        +test_custom_repr()
    }

    TestFailing --> otherfunc_multi
    TestMoreErrors --> somefunc

Summary

`failure_demo.py` is a deliberately failure-focused test suite that exercises many pytest assertion failure modes and exception handling scenarios. It is primarily a diagnostic or educational tool showcasing pytest’s detailed and user-friendly failure reports, rather than a functional test suite verifying application logic.

By exploring this file, developers and testers can gain insight into:

• How pytest compares complex data structures.

• How pytest reports exceptions and assertion errors.

• Best practices in writing tests that produce meaningful failure messages.

• The behavior of pytest in dynamic and runtime-generated test code.

End of Documentation for failure_demo.py