json.rs

Overview

The `json.rs` file provides a Rust-based fallback JSON deserialization backend that converts JSON text into Python objects. It is part of a larger JSON parsing system designed to optionally use a high-performance C-based parser (`yyjson`) when enabled, or this Rust fallback parser otherwise.

This file leverages the `serde_json` crate for parsing JSON strings and implements a custom visitor pattern (`JsonValue`) to recursively translate JSON primitives, arrays, and objects into Python-native data structures, represented as raw pointers (`NonNull`) to Python objects managed via the PyO3 FFI layer.

The fallback parser ensures broad compatibility and ease of maintenance when the `yyjson` feature flag is disabled, providing a robust deserialization path that integrates with error handling and the Python runtime environment.

Detailed Documentation

Functions

deserialize

pub(crate) fn deserialize(
    data: &'static str,
) -> Result<NonNull<pyo3_ffi::PyObject>, DeserializeError<'static>>

• Purpose:
  Entry point for deserializing a JSON string slice into a Python object pointer.

• Parameters:

  • data: &'static str — A static string slice containing the JSON text to parse.

• Returns:

  • Ok(NonNull<pyo3_ffi::PyObject>) — A non-null pointer to the root Python object representing the parsed JSON.

  • Err(DeserializeError) — An error describing why deserialization failed, including position information.

• Usage Example:

let json_text = r#"{"key": "value", "list": [1, 2, 3]}"#;
match deserialize(json_text) {
    Ok(py_obj_ptr) => {
        // Use py_obj_ptr via FFI in Python environment
    }
    Err(e) => {
        eprintln!("Deserialization failed: {}", e);
    }
}

• Implementation Details:

  • Uses serde_json::Deserializer to parse the JSON string.

  • Creates a JsonValue seed and calls its deserialize method.

  • On success, calls deserializer.end() to ensure full consumption of input.

  • Maps serde_json errors into DeserializeError with contextual info.

Structs

JsonValue

#[derive(Clone, Copy)]
struct JsonValue;

• Purpose:
  Implements the Serde DeserializeSeed and Visitor traits to recursively convert JSON data into Python objects.

• Key Traits Implemented:

  • DeserializeSeed<'de> — Provides a seed for deserializing JSON values.

  • Visitor<'de> — Handles visiting JSON data types (unit, bool, numbers, strings, sequences, maps).

• Methods:

deserialize

fn deserialize<D>(self, deserializer: D) -> Result<Self::Value, D::Error>
where
    D: Deserializer<'de>;

• Purpose:
  Entry point for deserialization that directs the Deserializer to parse any JSON type and visit it using this Visitor.

• Returns:

  • Result<NonNull<pyo3_ffi::PyObject>, D::Error> — The Python object pointer or an error.

expecting

fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result;

• Purpose:
  Formats a message describing what data this visitor expects. In this implementation, it returns an empty message (no expectation string).

Visitor Methods

These methods correspond to JSON types and convert them to Python objects:

• visit_unit — JSON null → Python None
  Calls parse_none() to create a Python None object.

• visit_bool — JSON boolean → Python bool
  Calls parse_bool(value).

• visit_i64 — JSON integer (signed) → Python int
  Calls parse_i64(value).

• visit_u64 — JSON integer (unsigned) → Python int
  Calls parse_u64(value).

• visit_f64 — JSON floating-point → Python float
  Calls parse_f64(value).

• visit_borrowed_str and visit_str — JSON string → Python str
  Converts JSON string to a Python Unicode string using PyStr::from_str(value).as_non_null_ptr().

• visit_seq — JSON array → Python list
  Deserializes each element recursively, collects pointers in a SmallVec, then creates a Python list and sets items.

• visit_map — JSON object → Python dict
  Iterates over key-value pairs, converts keys to Python Unicode strings, values recursively, then populates a Python dictionary.

Important Implementation Details

• Memory Safety and PyObject Pointers:
  The deserialized Python objects are represented as NonNull<pyo3_ffi::PyObject>, ensuring non-null raw pointers to Python objects managed by the PyO3 FFI. Care is taken to create and populate these objects using Python C-API functions (PyList_New, PyDict_New, PyList_SET_ITEM, etc.) wrapped in macros (ffi!, nonnull!, pydict_setitem!) for safety and convenience.

• SmallVec Usage:
  The visit_seq method uses SmallVec with an inline capacity of 8 to efficiently collect Python object pointers during array deserialization, minimizing heap allocations for small JSON arrays.

• Error Handling:
  Errors from serde_json parsing are converted into the crate's custom DeserializeError type, which carries error messages, line and column info, and the original JSON source string for diagnostics.

Interaction with Other System Components

• FFI Layer:
  The resulting Python objects from this deserialization process are intended to be used via the PyO3 FFI layer, bridging Rust and the Python runtime.

• Conditional Compilation:
  This file is used as the JSON deserialization backend when the yyjson feature is not enabled. When yyjson is enabled, a different backend (yyjson module) handles parsing.

• Error Handling:
  Integrates with the broader error management system by returning DeserializeError on failure, which is then handled or propagated to the Python layer.

• Serde Ecosystem:
  Relies heavily on Serde's deserialization traits and the serde_json crate for parsing JSON text.

Usage Example: Deserializing JSON Text to Python Object

use crate::deserialize::json::deserialize;

fn main() {
    let json_data = r#"{"name": "Alice", "age": 30, "likes": ["rust", "python"]}"#;

    match deserialize(json_data) {
        Ok(py_obj_ptr) => {
            // py_obj_ptr points to a Python dict representing the JSON data.
            // Further manipulation or FFI calls can be made here.
        }
        Err(err) => {
            eprintln!("Failed to parse JSON: {}", err);
        }
    }
}

Mermaid Diagram: Structure of json.rs

classDiagram
    class JsonValue {
        +deserialize<D>(deserializer: D) Result<NonNull<PyObject>, D::Error>
        +expecting(formatter: &mut fmt::Formatter) fmt::Result
        +visit_unit() Result<NonNull<PyObject>, E>
        +visit_bool(value: bool) Result<NonNull<PyObject>, E>
        +visit_i64(value: i64) Result<NonNull<PyObject>, E>
        +visit_u64(value: u64) Result<NonNull<PyObject>, E>
        +visit_f64(value: f64) Result<NonNull<PyObject>, E>
        +visit_borrowed_str(value: &str) Result<NonNull<PyObject>, E>
        +visit_str(value: &str) Result<NonNull<PyObject>, E>
        +visit_seq(seq: SeqAccess) Result<NonNull<PyObject>, E>
        +visit_map(map: MapAccess) Result<NonNull<PyObject>, E>
    }

    class json_rs {
        +deserialize(data: &str) Result<NonNull<PyObject>, DeserializeError>
    }

    json_rs --> JsonValue : uses

Summary

The `json.rs` file implements a robust Rust fallback JSON deserialization backend that transforms JSON text into Python objects using Serde's deserialization framework. It provides comprehensive coverage for all JSON data types, converting them into corresponding Python objects via the PyO3 FFI layer. This backend is integral to the JSON parsing subsystem when the `yyjson` feature is disabled, ensuring compatibility and maintainability without sacrificing correctness or error transparency. The design prioritizes safe pointer handling, efficient memory usage, and clear error propagation, making it a reliable core component of the JSON deserialization infrastructure.