test_source.py

Overview

The `test_source.py` file contains an extensive set of unit tests and utility functions designed to verify and demonstrate the functionality of the `Source` and related classes from the `_pytest._code` module. These classes provide sophisticated mechanisms for retrieving, parsing, and manipulating Python source code programmatically.

Primarily, the file tests how source code can be extracted from functions, methods, classes, and raw code strings, and how these sources can be indexed, stripped, deindented, and segmented into logical code statements. It also tests edge cases such as multiline strings, comments, decorators, single-line constructs, and error handling scenarios.

This test suite is critical for ensuring that pytest's internal source code handling features work correctly, which in turn supports accurate traceback reporting, source code introspection, and test reporting.

Detailed Explanation of Classes and Functions

Functions

test_source_str_function() -> None

• Purpose: Tests the string conversion (__str__) of Source objects created from various raw string inputs.

• Behavior: Asserts that leading whitespace is stripped correctly and multiline strings preserve newline characters.

• Usage Example:

  x = Source("   3")
  print(str(x))  # outputs: "3"
  

test_source_from_function() -> None

• Purpose: Tests creating a Source object from a Python function object.

• Behavior: Checks that the string representation of the source starts with the correct function definition header.

test_source_from_method() -> None

• Purpose: Tests creating a Source from a method bound to a class instance.

• Behavior: Verifies that the lines of code correspond exactly to the method's source lines.

test_source_from_lines() -> None

• Purpose: Tests creating a Source object directly from a list of source code lines.

• Behavior: Ensures trailing newlines are stripped from lines.

test_source_from_inner_function() -> None

• Purpose: Tests source extraction from an inner function (nested function).

• Behavior: Asserts the source string starts with the correct def statement.

test_source_strips() -> None

• Purpose: Tests the strip() method of Source objects.

• Behavior: Verifies that empty sources strip down to empty sources and equivalent sources remain equal.

test_source_strip_multiline() -> None

• Purpose: Tests multiline source stripping behavior.

• Behavior: Ensures empty or whitespace-only lines are removed properly.

Class TestAccesses

• Purpose: Tests Source object's indexing, slicing, length, and iteration capabilities.

• Key Methods:

  • setup_class(): Initializes a Source object with sample multi-line Python code.

  • test_getrange(): Asserts slicing returns the correct lines of source.

  • test_getrange_step_not_supported(): Confirms that slicing with a step raises an IndexError.

  • test_getline(): Tests getting a single line by index.

  • test_len(): Verifies the length of the source in lines.

  • test_iter(): Checks iteration over source lines.

Class TestSourceParsing

• Purpose: Tests advanced parsing and statement extraction functionality.

• Key Methods:

  • setup_class(): Initializes a multi-line source including complex expressions.

  • test_getstatement(): Tests retrieval of complete statements spanning multiple lines.

  • test_getstatementrange_triple_quoted(): Checks statement range detection on triple-quoted strings.

  • test_getstatementrange_within_constructs(): Tests statement range detection within nested try-except-finally blocks.

  • test_getstatementrange_bug and test_getstatementrange_bug2(): Verify fixes for past bugs related to statement range detection.

  • test_getstatementrange_ast_issue58(): Tests statement range detection related to a specific AST issue.

  • test_getstatementrange_out_of_bounds_py3(): Handles out-of-bounds line numbers gracefully.

  • test_getstatementrange_with_syntaxerror_issue7(): Ensures SyntaxError is raised for invalid source.

Other Key Functional Tests

• test_getstartingblock_singleline(): Tests retrieval of source for a single-line block.

• test_getline_finally(): Verifies source retrieval in try-finally blocks with exceptions.

• test_getfuncsource_dynamic(): Tests source extraction from dynamically defined functions.

• test_getfuncsource_with_multiline_string(): Ensures multiline strings in functions are preserved correctly.

• test_deindent(): Tests the deindent utility, which removes leading tabs or spaces to normalize indentation.

• test_source_of_class_at_eof_without_newline(): Tests source retrieval of classes at end of file without trailing newline.

• test_source_fallback(), test_findsource_fallback(), test_findsource(): Tests fallback mechanisms for retrieving source when standard methods fail.

• test_getfslineno(): Tests retrieving file system path and line number of source objects.

• test_code_of_object_instance_with_call(): Tests behavior when Source or Code is constructed from callable instances.

• getstatement(lineno: int, source) -> Source: Utility function to get the source statement corresponding to a particular line number.

• Several tests (test_oneline(), test_comment_and_no_newline_at_end(), test_comments(), etc.) test statement extraction in various contexts including comments, semicolons, decorators, and multiline expressions.

Classes Testing Specific Python Constructs

• TestTry, TestTryFinally, TestIf: These classes test statement extraction in control flow constructs such as try/except/else/finally and if/elif/else, ensuring correct source mapping to each block.

Important Implementation Details and Algorithms

• The tests rely heavily on the Source class and related utilities (Code, Frame, getfslineno, Source.getstatement, etc.) from _pytest._code.

• The Source class abstracts a block of Python source code, supporting indexing, iteration, and slicing, and provides methods to extract logical statements spanning multiple lines, even in the presence of complex syntax like multiline strings or nested blocks.

• Statement extraction uses AST-based techniques (getstatementrange_ast) to determine the start and end lines of a statement given a line number.

• Deindenting is carefully handled to normalize indentation irrespective of tabs or spaces.

• The tests check robustness against edge cases such as missing newlines at EOF, syntax errors, dynamic code objects, and decorated functions.

• The fallback logic tested involves using linecache and compile to retrieve source when the standard introspection methods fail, ensuring maximum resilience in source retrieval.

Interaction with Other Parts of the System

• This test file is part of the pytest codebase and directly tests core source inspection utilities located in _pytest._code.source and related modules.

• It indirectly supports pytest's ability to:

  • Display accurate source code snippets in tracebacks and test failure reports,

  • Provide source-aware features like skipping tests based on source markers,

  • Support code coverage tools and debugging aids.

• The use of pytest fixtures, markers, and the test framework itself shows integration with pytest's testing infrastructure.

• The file imports utilities like import_path from _pytest.pathlib to dynamically import modules during tests.

• The tests also indirectly validate the behavior of Python's introspection APIs (inspect, linecache) as wrapped by pytest utilities.

Visual Diagram: Class and Function Structure

flowchart TD
    A[Tests in test_source.py]

    A --> B[test_source_str_function()]
    A --> C[test_source_from_function()]
    A --> D[test_source_from_method()]
    A --> E[test_source_from_lines()]
    A --> F[test_source_from_inner_function()]
    A --> G[test_source_strips()]
    A --> H[test_source_strip_multiline()]
    A --> I[TestAccesses]
    A --> J[TestSourceParsing]
    A --> K[test_getstartingblock_singleline()]
    A --> L[test_getline_finally()]
    A --> M[test_getfuncsource_dynamic()]
    A --> N[test_getfuncsource_with_multiline_string()]
    A --> O[test_deindent()]
    A --> P[test_source_of_class_at_eof_without_newline()]
    A --> Q[test_source_fallback()]
    A --> R[test_findsource_fallback()]
    A --> S[test_findsource()]
    A --> T[test_getfslineno()]
    A --> U[test_code_of_object_instance_with_call()]
    A --> V[getstatement()]
    A --> W[TestTry]
    A --> X[TestTryFinally]
    A --> Y[TestIf]
    A --> Z[Other test functions (comments, semicolon, decorators, etc.)]

    subgraph TestAccesses
        I1[setup_class()]
        I2[test_getrange()]
        I3[test_getrange_step_not_supported()]
        I4[test_getline()]
        I5[test_len()]
        I6[test_iter()]
    end

    subgraph TestSourceParsing
        J1[setup_class()]
        J2[test_getstatement()]
        J3[test_getstatementrange_triple_quoted()]
        J4[test_getstatementrange_within_constructs()]
        J5[test_getstatementrange_bug()]
        J6[test_getstatementrange_bug2()]
        J7[test_getstatementrange_ast_issue58()]
        J8[test_getstatementrange_out_of_bounds_py3()]
        J9[test_getstatementrange_with_syntaxerror_issue7()]
    end

    I --> I1
    I --> I2
    I --> I3
    I --> I4
    I --> I5
    I --> I6

    J --> J1
    J --> J2
    J --> J3
    J --> J4
    J --> J5
    J --> J6
    J --> J7
    J --> J8
    J --> J9

Summary

`test_source.py` is a highly comprehensive test suite validating the extraction, parsing, and handling of Python source code by pytest's internal utilities. It covers a wide range of real-world scenarios and edge cases, ensuring that pytest can reliably introspect and manipulate source code for improved test reporting, debugging, and dynamic behavior. The tests leverage Python introspection capabilities and robust fallback mechanisms to maximize reliability across diverse environments and code structures.