test_ujson.py

Overview

`test_ujson.py` is a comprehensive test suite designed to verify the functionality, correctness, and robustness of JSON serialization and deserialization using the `orjson` library. The tests focus on ensuring that `orjson` can correctly encode various Python data types (including integers, floats, strings, lists, dictionaries, booleans, and `None`) into JSON, and decode JSON back into Python objects without loss of fidelity.

This file aims to:

• Validate encoding and decoding of primitive types, complex nested structures, and Unicode strings.

• Check handling of edge cases such as very large numbers, decimal precision, Unicode surrogate pairs, control characters, and null characters.

• Ensure that error conditions and malformed JSON input raise appropriate exceptions.

• Compare certain behaviors with the standard json module to confirm compatibility.

The file uses `pytest` as the testing framework and heavily relies on `orjson.dumps` and `orjson.loads` for JSON operations.

Classes and Methods

Class: TestUltraJSON

This class encapsulates all test cases for JSON serialization/deserialization using `orjson`. Each method tests a specific scenario or data type.

Test Methods

1. test_doubleLongIssue(self)

• Purpose: Tests encoding and decoding of a dictionary containing a very large negative integer.

• Parameters: None

• Returns: None

• Behavior:

  • Serializes a dict with a large negative long integer.

  • Deserializes it back and asserts equality.

  • Repeats the process twice.

• Example:

  sut = {"a": -4342969734183514}
  encoded = orjson.dumps(sut)
  decoded = orjson.loads(encoded)
  assert sut == decoded
  

2. test_doubleLongDecimalIssue(self)

• Purpose: Tests encoding and decoding of a dictionary with a large floating-point number.

• Details: Similar to the previous test but with a large decimal floating-point value.

3. test_encodeDecodeLongDecimal(self)

• Purpose: Tests encoding and decoding of a dictionary with a negative decimal float.

• Note: Does not assert explicitly but checks for successful encode/decode without exceptions.

4. test_decimalDecodeTest(self)

• Purpose: Tests approximate equality of floating-point numbers after encode/decode.

• Implementation: Uses pytest.approx to compare floating values.

5. test_encodeDictWithUnicodeKeys(self)

• Purpose: Tests encoding of dictionaries with repeated keys and Unicode keys.

• Details: Demonstrates that orjson.dumps can handle Unicode keys and repeated keys (though dictionaries inherently overwrite duplicate keys).

6. test_encodeArrayOfNestedArrays(self)

• Purpose: Tests encoding and decoding of deeply nested arrays repeated multiple times.

• Details: Ensures nested structures are preserved intact.

7. test_encodeArrayOfDoubles(self)

• Purpose: Tests encoding/decoding of arrays of floating-point numbers repeated multiple times.

8. test_encodeStringConversion2(self)

• Purpose: Tests encoding and decoding of strings containing backslashes, slashes, and control characters.

• Checks:

  • Correct escaping in JSON bytes output.

  • Lossless round-trip conversion.

9. test_decodeUnicodeConversion(self)

• Purpose: Placeholder test, currently no implementation.

10. test_encodeUnicodeConversion1(self)

• Purpose: Tests encoding and decoding of strings containing extended Unicode characters, including Arabic script.

• Checks:

  • Consistency of encoded bytes and decoded strings.

11. test_encodeControlEscaping(self)

• Purpose: Tests encoding and decoding of control characters like \x19.

• Checks: Round-trip consistency.

12-16. Various test_encodeUnicode... methods

• Purpose: Test encoding of different Unicode byte sequences, surrogate pairs, and 4-byte UTF-8 sequences.

• Example: Ensures emoji and high Unicode plane characters serialize correctly.

17. testEncodeUnicodeBMP(self)

• Purpose: Tests encoding BMP (Basic Multilingual Plane) Unicode characters (emojis).

• Comparison: Checks equivalence with Python's standard json module.

18. testEncodeSymbols(self)

• Purpose: Tests encoding/decoding of special symbols like ✿♡✿.

• Checks:

  • orjson produces UTF-8 encoded bytes equivalent to json.dumps with ensure_ascii=False.

19-26. Various tests for arrays, integers (positive and negative), long integers, lists, dictionaries, None, booleans.

• Purpose: Verify proper behavior of basic JSON data types serialization and deserialization.

27. test_encodeToUTF8(self)

• Purpose: Tests encoding and decoding of UTF-8 decoded bytes string.

28. test_decodeFromUnicode(self)

• Purpose: Tests decoding JSON strings from both str and Unicode string.

29-35. Various tests for error conditions in decoding:

• Purpose: Confirm that malformed or incomplete JSON strings raise orjson.JSONDecodeError.

• Examples: Invalid tokens, broken arrays/objects, unterminated strings, broken boolean/null literals.

36-38. Leak tests for broken dict keys and lists.

• Purpose: Repeatedly attempt to decode malformed JSON to check for memory leaks or stability.

39-41. Additional malformed JSON tests raising exceptions.

42-43. Numeric decoding tests for positive and negative integers.

44-51. Null character encoding/decoding tests.

• Purpose: Ensure null characters within strings are correctly encoded as \u0000 and decoded properly.

52-54. Escape sequence decoding tests.

• Purpose: Confirm that escape sequences in strings decode identically in orjson and json modules, including large escaped strings.

Important Implementation Details

• The test suite uses the orjson library, a fast JSON parser and serializer implemented in Rust with Python bindings.

• Tests cover a broad spectrum of JSON features including:

  • Unicode handling (BMP and supplementary planes).

  • Control and null characters escaping.

  • Large integers and floating-point precision.

  • Complex nested arrays and dictionaries.

• The suite uses pytest's exception-catching mechanism (pytest.raises) to verify that invalid JSON inputs raise appropriate exceptions.

• Many tests perform repeated encode-decode cycles to confirm consistency and idempotency.

• Some tests compare orjson behavior with Python’s built-in json module to ensure compatibility or highlight differences.

• There are tests designed to detect potential memory leaks or internal parser state corruption by repeatedly processing malformed inputs.

Interaction with Other Parts of the System

• This file is a test module primarily interacting with:

  • The orjson library for JSON operations.

  • The pytest testing framework for test discovery and assertions.

  • The standard json module for cross-validation in some tests.

• It does not define any new reusable components but verifies the behavior of orjson integration.

• Likely part of a larger testing framework or CI pipeline that validates JSON serialization features before deployment or release.

Usage

To run the tests in this file, ensure that `pytest` and `orjson` are installed:

pip install pytest orjson
pytest test_ujson.py

Visual Diagram

classDiagram
    class TestUltraJSON {
        +test_doubleLongIssue()
        +test_doubleLongDecimalIssue()
        +test_encodeDecodeLongDecimal()
        +test_decimalDecodeTest()
        +test_encodeDictWithUnicodeKeys()
        +test_encodeArrayOfNestedArrays()
        +test_encodeArrayOfDoubles()
        +test_encodeStringConversion2()
        +test_decodeUnicodeConversion()
        +test_encodeUnicodeConversion1()
        +test_encodeControlEscaping()
        +test_encodeUnicodeConversion2()
        +test_encodeUnicodeSurrogatePair()
        +test_encodeUnicode4BytesUTF8()
        +test_encodeUnicode4BytesUTF8Highest()
        +testEncodeUnicodeBMP()
        +testEncodeSymbols()
        +test_encodeArrayInArray()
        +test_encodeIntConversion()
        +test_encodeIntNegConversion()
        +test_encodeLongNegConversion()
        +test_encodeListConversion()
        +test_encodeDictConversion()
        +test_encodeNoneConversion()
        +test_encodeTrueConversion()
        +test_encodeFalseConversion()
        +test_encodeToUTF8()
        +test_decodeFromUnicode()
        +test_decodeJibberish()
        +test_decodeBrokenArrayStart()
        +test_decodeBrokenObjectStart()
        +test_decodeBrokenArrayEnd()
        +test_decodeBrokenObjectEnd()
        +test_decodeObjectDepthTooBig()
        +test_decodeStringUnterminated()
        +test_decodeStringUntermEscapeSequence()
        +test_decodeStringBadEscape()
        +test_decodeTrueBroken()
        +test_decodeFalseBroken()
        +test_decodeNullBroken()
        +test_decodeBrokenDictKeyTypeLeakTest()
        +test_decodeBrokenDictLeakTest()
        +test_decodeBrokenListLeakTest()
        +test_decodeDictWithNoKey()
        +test_decodeDictWithNoColonOrValue()
        +test_decodeDictWithNoValue()
        +test_decodeNumericIntPos()
        +test_decodeNumericIntNeg()
        +test_encodeNullCharacter()
        +test_decodeNullCharacter()
        +test_decodeEscape()
        +test_decodeBigEscape()
    }

Summary

`test_ujson.py` is a detailed and thorough test suite validating the correctness and robustness of JSON serialization and parsing via the `orjson` library. It covers common use cases, edge cases, Unicode intricacies, and error handling, ensuring that `orjson` behaves as expected in a wide range of scenarios. The file is critical for maintaining the integrity of JSON handling in any application relying on `orjson` for performance and accuracy.