Rust Deserialization Backend

Purpose

The Rust Deserialization Backend addresses the essential task of converting raw JSON text into Python objects within the core JSON parsing module. It provides a modular and flexible deserialization mechanism that conditionally uses either a high-performance embedded C JSON parser (`yyjson`) or a pure Rust fallback parser. This design ensures robustness and adaptability: when the `yyjson` feature is enabled, the backend leverages optimized parsing paths; otherwise, it falls back to a reliable serde-based Rust parser. This backend is crucial for transforming JSON input into Python-native data structures while handling errors and edge cases gracefully.

Functionality

The backend exposes a unified `deserialize` function that takes JSON text and returns a pointer to a Python object representing the parsed data. Internally, it selects the parser implementation based on compile-time features:

With yyjson enabled: It uses the yyjson module, which integrates the C library for ultra-fast parsing.
Without yyjson: It employs the Rust serde_json crate as a fallback, implementing detailed deserialization logic via Serde traits.

The fallback JSON parser (`json.rs`) uses Serde's [Deserializer](/projects/287/67745) and a custom `Visitor` (`JsonValue`) to recursively convert JSON primitives into Python objects:

Primitive Types: JSON null, booleans, integers, floats, and strings map directly to Python None, bool, int, float, and str objects.
Arrays: Elements are deserialized sequentially and collected into a Python list.
Objects: Key-value pairs are deserialized into a Python dictionary, converting JSON string keys into Python Unicode objects.

This visitor pattern ensures comprehensive coverage of JSON data types and proper error reporting through `DeserializeError`.

Example snippet illustrating the deserialization dispatch:

#[cfg(feature = "yyjson")]
pub(crate) use yyjson::deserialize;

#[cfg(not(feature = "yyjson"))]
pub(crate) use json::deserialize;

Example of deserializing a JSON array into a Python list:

fn visit_seq<A>(self, mut seq: A) -> Result<Self::Value, A::Error>
where
    A: SeqAccess<'de>,
{
    // Collect elements into a SmallVec for efficient temporary storage
    let mut elements: SmallVec<[*mut pyo3_ffi::PyObject; 8]> = SmallVec::with_capacity(8);
    while let Some(elem) = seq.next_element_seed(self)? {
        elements.push(elem.as_ptr());
    }
    let ptr = ffi!(PyList_New(elements.len() as isize));
    for (i, &obj) in elements.iter().enumerate() {
        ffi!(PyList_SET_ITEM(ptr, i as isize, obj));
    }
    Ok(nonnull!(ptr))
}

Integration and Relationship

This backend is a core submodule of the **High-Performance JSON Parsing** main topic, focusing specifically on the deserialization path. It complements the **Embedded yyjson Integration** subtopic by conditionally relying on the `yyjson` parser when available, or falling back to its own Rust-based parser otherwise.

Together with serialization components in the broader Rust core, this backend forms the bridge between raw JSON data and Python objects. It integrates tightly with:

FFI Layer: The deserialized Python objects produced here are passed through the FFI bindings to be exposed to Python code.
Error Handling: Errors raised during deserialization are wrapped into custom error types documented under the Error Handling and Exceptions topic.
Benchmarking: This backend’s performance directly impacts the deserialization benchmarks executed in the suite, influencing optimization priorities.

By providing a clean abstraction over multiple parsing implementations, it enables the project to balance speed and compatibility without forcing a single parser choice.

Diagram

flowchart TD
    Input[JSON Text Input]
    CheckFeature{Is yyjson Feature Enabled?}
    YYJSONParser[yyjson C Parser Module]
    RustFallback[Rust serde_json Parser Module]
    DeserializeFunc[deserialize() Function]
    PyObject[Python Object Output]
    ErrorHandle[DeserializeError Handling]

    Input --> DeserializeFunc
    DeserializeFunc --> CheckFeature
    CheckFeature -->|Yes| YYJSONParser
    CheckFeature -->|No| RustFallback
    YYJSONParser --> PyObject
    RustFallback --> PyObject
    YYJSONParser --> ErrorHandle
    RustFallback --> ErrorHandle

This flowchart visualizes the deserialization decision process, showing how input JSON text is routed to the appropriate parsing backend based on compilation features, culminating in Python object outputs or error reporting.