test_transform.py

Overview

`test_transform.py` is a test suite module designed to validate the transformation and parsing behavior of JSON data using the `orjson` library. The file primarily focuses on verifying that JSON inputs are correctly decoded and then re-encoded (serialized) to the expected output forms, ensuring compliance with JSON specifications and handling edge cases such as numeric representations, key normalization, and invalid Unicode code points.

This file contains a collection of automated tests implemented using the `pytest` framework. The tests utilize fixture data representing various JSON inputs and check both successful transformations and failure cases where JSON decoding should raise errors.

Detailed Explanation

Helper Functions

_read_file(filename)

• Purpose:
  Reads a binary fixture file from the "transform" directory using a utility function and strips trailing newline and carriage return bytes.

• Parameters:

  • filename (str): The name of the JSON fixture file to read.

• Returns:

  • bytes: The raw bytes of the JSON file content, cleaned of trailing newlines/carriage returns.

• Usage Example:

  data = _read_file("number_1e6.json")
  

• Implementation Details:
  Utilizes read_fixture_bytes from the module's util package, which abstracts the file reading and test fixture handling.

Class: TestJSONTestSuiteTransform

This class encapsulates multiple test cases for JSON transformation validation. It is decorated with `@needs_data`, indicating tests require external fixture data to run.

Methods:

_pass_transform(self, filename, reference=None)

• Purpose:
  Tests that a JSON file can be loaded with orjson.loads and then dumped back to JSON bytes with orjson.dumps, matching either the original data or a provided reference output.

• Parameters:

  • filename (str): The JSON fixture filename to test.

  • reference (bytes, optional): Expected JSON bytes output after serialization. If not provided, the original input data is used as the reference.

• Returns:

  • None. Asserts equality of transformed data.

• Usage Example:

  self._pass_transform("number_1e6.json", b"[1000000.0]")
  

• Implementation Details:
  Reads the file, loads JSON, dumps it again, and asserts the output matches expected bytes.

_fail_transform(self, filename)

• Purpose:
  Tests that loading a JSON file with orjson.loads raises a JSONDecodeError, indicating invalid JSON data.

• Parameters:

  • filename (str): The JSON fixture filename expected to fail decoding.

• Returns:

  • None. Asserts that an exception is raised.

• Usage Example:

  self._fail_transform("string_1_invalid_codepoint.json")
  

• Implementation Details:
  Uses pytest.raises context manager to check for exceptions.

Test Methods

Each test method corresponds to a specific JSON fixture file, named descriptively to indicate the focus of the test (e.g., numeric formats, object keys, string escapes). The methods either call `_pass_transform` or `_fail_transform` depending on whether the input is expected to be valid or invalid JSON.

• Examples:

  • test_number_1(self)
    Validates the file number_1.0.json passes transformation.

  • test_number_1e6(self)
    Validates number_1e6.json transforms to [1000000.0].

  • test_string_1_invalid_codepoint(self)
    Validates string_1_invalid_codepoint.json raises a decode error.

• Special Cases:

  • test_number_10000000000000000999
    This test reads a large number that cannot be serialized by orjson.dumps but can be loaded successfully, so it only asserts the loaded value.

  • test_object_same_key_unclear_values
    This test asserts that the loaded data matches one of two possible outputs, acknowledging backend-specific differences in handling duplicate keys.

Important Implementation Details

• Data Handling:
  Test inputs are read from external fixture files, presumably JSON files stored under a transform directory. The read_fixture_bytes utility abstracts the path and file reading.

• JSON Parsing and Serialization:
  The module uses orjson for both parsing (orjson.loads) and serialization (orjson.dumps), taking advantage of its performance and strict compliance.

• Error Handling:
  Tests explicitly check for orjson.JSONDecodeError when invalid JSON inputs are encountered, ensuring the parser correctly rejects malformed data.

• Unicode and Normalization Testing:
  Tests verify normalization of object keys in NFC and NFD forms, as well as handling of invalid Unicode code points in strings.

Integration with Other System Components

• The file depends on:

  • pytest: For running test cases and handling assertion and exception verification.

  • orjson: The JSON processing library under test.

  • .util: A local utility module providing needs_data (likely a decorator to check for presence of fixture data) and read_fixture_bytes (for loading test fixtures).

• This test module is part of a larger test suite validating the JSON handling capabilities of the system or library that uses orjson.

• The fixtures it tests against are external JSON files stored in the test data directory, making it highly data-driven and modular for adding more test cases.

Visual Diagram

classDiagram
    class TestJSONTestSuiteTransform {
        +_pass_transform(filename: str, reference: bytes = None)
        +_fail_transform(filename: str)
        +test_number_1()
        +test_number_1e6()
        +test_number_1e_999()
        +test_number_10000000000000000999()
        +test_number_1000000000000000()
        +test_object_key_nfc_nfd()
        +test_object_key_nfd_nfc()
        +test_object_same_key_different_values()
        +test_object_same_key_same_value()
        +test_object_same_key_unclear_values()
        +test_string_1_escaped_invalid_codepoint()
        +test_string_1_invalid_codepoint()
        +test_string_2_escaped_invalid_codepoints()
        +test_string_2_invalid_codepoints()
        +test_string_3_escaped_invalid_codepoints()
        +test_string_3_invalid_codepoints()
        +test_string_with_escaped_NULL()
    }
    class _read_file {
        +filename: str
        +return bytes
    }
    TestJSONTestSuiteTransform ..> _read_file : uses

Summary

`test_transform.py` is a focused test module for validating JSON transformations using `orjson`. It reads numerous JSON fixtures covering numeric edge cases, key normalization, and invalid Unicode sequences, ensuring both correct serialization and proper error detection. It leverages utility functions for fixture management and integrates tightly with pytest for automated test execution. This module is essential for maintaining the correctness and robustness of the JSON processing capabilities in the larger software system.