lib.rs

Overview

`lib.rs` serves as the core Rust source file for the `orjson` Python extension module, which provides high-performance JSON serialization and deserialization capabilities. This file primarily handles the integration between Rust and Python by defining the Python module, exposing the serialization (`dumps`) and deserialization (`loads`) functions to Python, and managing error handling relevant to JSON processing.

The module leverages Rust's performance and safety features while interfacing directly with Python's C API (through `pyo3_ffi`) to create a seamless, efficient JSON processing backend. It also exposes several module-level constants and custom error types.

Detailed Breakdown

Modules and Imports

• Internal Modules:

  • util: Utility functions and macros.

  • alloc: Memory allocation helpers.

  • deserialize: Handles JSON deserialization logic.

  • serialize: Handles JSON serialization logic.

  • ffi: Foreign Function Interface helpers for Python C API.

  • opt: Definitions of serialization options.

  • str: String helpers.

  • typeref: References to Python types and exceptions used in the module.

• External crates:

  • pyo3_ffi: Raw bindings to Python C API functions and structures.

  • core: Rust core library for low-level operations.

Macros

• add!: Adds Python objects (functions, constants, types) to the Python module, adapting to Python version differences.

• opt!: Adds integer constants representing option flags to the module, with platform-specific handling.

• raise_*_exception: Helper functions to raise Python exceptions from Rust code.

Key Functions

orjson_init_exec

unsafe extern "C" fn orjson_init_exec(mptr: *mut PyObject) -> c_int

• Purpose:
  Initializes the orjson Python module during import. It registers functions, constants, types, and error classes with the Python interpreter.

• Parameters:

  • mptr: Pointer to the Python module object.

• Returns:

  • c_int: Returns 0 on successful initialization.

• Behavior:

  • Initializes type references.

  • Sets the __version__ attribute.

  • Defines and adds the dumps and loads Python functions.

  • Adds the Fragment type and various option constants for serialization behavior.

  • Adds custom exceptions JSONDecodeError and JSONEncodeError.

• Usage Example (Python):

  import orjson
  print(orjson.__version__)
  data = orjson.dumps({"key": "value"})
  obj = orjson.loads(data)
  

PyInit_orjson

unsafe extern "C" fn PyInit_orjson() -> *mut PyModuleDef

• Purpose:
  Entry point called by Python to initialize the orjson module.

• Returns:

  • Pointer to a PyModuleDef struct representing the module.

• Behavior:

  • Creates module slots specifying initialization and metadata.

  • Creates and returns the module definition with associated slots.

• Implementation Detail:
  Uses conditional compilation to adapt to Python versions (3.10, 3.12, 3.13) for module definition slots.

loads

unsafe extern "C" fn loads(_self: *mut PyObject, obj: *mut PyObject) -> *mut PyObject

• Purpose:
  Deserializes a JSON-encoded Python object (obj) into an equivalent Python object.

• Parameters:

  • _self: Unused, placeholder for Python self pointer.

  • obj: Python object containing JSON data to deserialize.

• Returns:

  • Pointer to the deserialized Python object on success.

  • null_mut() on failure after setting a Python exception.

• Implementation Detail:
  Calls deserialize::deserialize and converts errors into Python exceptions using raise_loads_exception.

• Usage Example (Python):

  obj = orjson.loads(b'{"name":"Alice","age":30}')
  print(obj["name"])  # Output: Alice
  

dumps

unsafe extern "C" fn dumps(
    _self: *mut PyObject,
    args: *const *mut PyObject,
    nargs: Py_ssize_t,
    kwnames: *mut PyObject,
) -> *mut PyObject

• Purpose:
  Serializes a Python object into JSON bytes.

• Parameters:

  • _self: Unused, placeholder for Python self pointer.

  • args: Pointer to an array of Python objects representing positional arguments.

  • nargs: Number of positional arguments.

  • kwnames: Tuple of keyword argument names or null.

• Returns:

  • Pointer to a Python bytes object containing JSON data on success.

  • null_mut() on failure after setting a Python exception.

• Behavior:

  • Parses positional and keyword arguments for:

    • obj: mandatory positional argument to serialize.

    • default: optional callable for serializing unsupported types.

    • option: optional integer bitflags controlling serialization behavior.

  • Validates arguments and options.

  • Calls serialize::serialize to perform actual serialization.

  • Converts serialization errors into Python exceptions via raise_dumps_exception_dynamic.

• Usage Example (Python):

  json_bytes = orjson.dumps({"key": "value"}, option=orjson.OPT_INDENT_2)
  print(json_bytes.decode())  # Pretty-printed JSON string
  

Error Raising Helper Functions

• raise_loads_exception(err: DeserializeError): Converts deserialization errors into Python JSONDecodeError.

• raise_dumps_exception_fixed(msg: &str): Raises Python JSONEncodeError with a fixed error message.

• raise_dumps_exception_dynamic(err: &str): Raises Python JSONEncodeError with dynamic cause chaining, depending on Python version.

Important Implementation Details and Algorithms

• Version Compatibility:
  The module is carefully adapted for multiple Python versions (3.10, 3.12, 3.13) with conditional compilation directives. For example, macros and module slots change depending on the Python version.

• Low-Level FFI Usage:
  Uses pyo3_ffi to call Python C API directly for performance and fine control. This includes manual reference counting, creating Python tuples, strings, and raising exceptions.

• Option Flags:
  Serialization options are represented as bitflags (opt::Opt) and exposed as integer constants in the Python module, enabling flexible JSON output customization.

• Error Handling:
  Converts Rust errors into Python exceptions with detailed messages and preserves exception cause chains where possible.

• Unsafe Code:
  The file uses extensive unsafe blocks for FFI calls, pointer manipulation, and manual memory management, which is necessary for Python extension development in Rust.

Interaction with Other Parts of the System

• deserialize Module:
  Handles the core logic of JSON parsing into Python objects, invoked by the loads function.

• serialize Module:
  Handles the conversion of Python objects to JSON bytes, invoked by the dumps function.

• typeref Module:
  Provides references to Python types and exceptions used for error handling and type checking.

• opt Module:
  Defines option bitflags controlling serialization behavior.

• util Module:
  Contains utility functions like safe pointer casts and conversions.

• Python Interpreter:
  This file defines the interface and lifecycle of the orjson Python module within the interpreter.

Visual Diagram

The following Mermaid class diagram illustrates the main functions and their relationships in this utility-style file, focusing on FFI functions exposed to Python and error handling helpers:

classDiagram
    class lib {
        <<module>>
        +orjson_init_exec(mptr: *mut PyObject) c_int
        +PyInit_orjson() *mut PyModuleDef
        +loads(_self: *mut PyObject, obj: *mut PyObject) *mut PyObject
        +dumps(_self: *mut PyObject, args: *const *mut PyObject, nargs: Py_ssize_t, kwnames: *mut PyObject) *mut PyObject
        -raise_loads_exception(err: DeserializeError) *mut PyObject
        -raise_dumps_exception_fixed(msg: &str) *mut PyObject
        -raise_dumps_exception_dynamic(err: &str) *mut PyObject
    }
    
    lib --> deserialize : uses
    lib --> serialize : uses
    lib --> typeref : uses
    lib --> opt : uses
    lib --> util : uses

Summary

`lib.rs` is the core integration layer of the `orjson` Rust-implemented Python module, responsible for initializing the module, exposing key JSON serialization/deserialization functions, managing Python exceptions, and defining module constants. It bridges Rust's efficient JSON processing with Python's runtime via low-level FFI, carefully handling version compatibility and error propagation.

This file does not contain the core serialization or deserialization logic itself but delegates those to the respective `serialize` and `deserialize` modules, focusing on the Python interface, argument parsing, and error communication.

End of Documentation for lib.rs