fragment.rs

Overview

The [fragment.rs](/projects/287/67788) file provides functionality for serializing a **Fragment**, a low-level Python object wrapper, into a byte array using the Serde serialization framework in Rust. It defines the `FragmentSerializer` struct which wraps a raw Python object pointer and implements the `Serialize` trait for it. The serialization logic extracts the underlying byte data from the Python object, handling either Python bytes or string types, and converts this data into a format compatible with Serde serializers.

This file serves as a bridge between Python's internal byte/string objects and Rust's serialization ecosystem, enabling efficient and safe serialization of Python data fragments in Rust applications that interface with Python code (likely through PyO3 or similar FFI layers).

Detailed Documentation

Struct: FragmentSerializer

pub(crate) struct FragmentSerializer {
    ptr: *mut pyo3_ffi::PyObject,
}

• Purpose: Wraps a raw pointer to a Python FFI object (PyObject) representing a Fragment.

• Visibility: pub(crate) — public within the current crate only.

• Field:

  • ptr: A mutable raw pointer to a Python object (PyObject), assumed to be a Fragment.

Methods

new

pub fn new(ptr: *mut pyo3_ffi::PyObject) -> Self

• Description: Constructs a new FragmentSerializer instance from a raw Python object pointer.

• Parameters:

  • ptr: Pointer to a Python object (PyObject) that wraps the fragment data.

• Returns: A new FragmentSerializer instance wrapping the given pointer.

• Example:

let py_fragment_ptr: *mut pyo3_ffi::PyObject = get_fragment_from_python();
let serializer = FragmentSerializer::new(py_fragment_ptr);

Trait Implementation: Serialize for FragmentSerializer

Implements the Serde `Serialize` trait, enabling `FragmentSerializer` to be serialized by any Serde-compatible serializer.

fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
    S: Serializer,

• Functionality:

  • Safely inspects the underlying Python fragment object.

  • Checks if the fragment contents are of Python bytes or string type.

  • Extracts the raw byte slice from the Python object:

    • If bytes: directly extracts the byte buffer.

    • If string: converts the Python string to UTF-8 bytes.

  • Returns an error if the fragment is neither bytes nor string.

  • Serializes the extracted byte slice using the provided Serde serializer.

• Error Handling:

  • Returns SerializeError::InvalidStr if the string conversion fails.

  • Returns SerializeError::InvalidFragment if the underlying object type is unsupported.

• Attributes:

  • #[cold] and #[inline(never)] hint to the compiler that this function is unlikely to be hot code, optimizing for size.

  • #[cfg_attr(feature = "optimize", optimize(size))] applies size optimization when the optimize feature is enabled.

• Example Usage:

use serde_json::to_vec;

let serializer = FragmentSerializer::new(py_fragment_ptr);
let serialized_bytes = to_vec(&serializer)?;

Implementation Details and Algorithms

• The FragmentSerializer holds a raw pointer to Python's PyObject and casts it unsafely to Fragment for internal access.

• Uses raw pointer checks (core::ptr::eq) to determine the Python object's type by comparing with known static type references (BYTES_TYPE, STR_TYPE).

• When identifying a bytes object, it uses FFI functions PyBytes_AS_STRING and PyBytes_GET_SIZE to obtain a raw pointer and length of the byte buffer.

• For string objects, it uses a safe wrapper PyStr::from_ptr_unchecked to obtain a Rust string slice, then converts to UTF-8 bytes.

• The extracted byte slice is then handed off to the Serde serializer's serialize_bytes method.

• Errors are propagated using macros (err!) that create Serde-compatible serialization errors from internal SerializeError variants.

This approach minimizes memory copying by directly referencing Python's internal buffers, which is efficient but requires careful unsafe code to avoid undefined behavior.

Interaction with Other System Components

• FFI Layer (crate::ffi): Provides the definitions and helper functions to interface with Python's C API (e.g., PyBytes_AS_STRING, PyBytes_GET_SIZE).

• Serialization Layer (serde::ser): Integrates with Serde's serialization traits to output serialized data in various formats (JSON, CBOR, etc.).

• Python Type References (crate::typeref): Provides static type pointers (BYTES_TYPE, STR_TYPE) used for runtime type checking.

• String Utilities (crate::str::PyStr): Wraps Python string objects and provides safe string extraction.

• Error Handling (crate::serialize::error): Defines serialization errors specific to this module.

• Utility Conversion (crate::util::isize_to_usize): Safely converts signed size types to unsigned sizes for Rust slice operations.

The `FragmentSerializer` is designed to be used wherever a Python fragment needs to be serialized, likely in contexts where Python data is passed through Rust code, serialized, and sent over the network or stored.

Visual Diagram

classDiagram
    class FragmentSerializer {
        - ptr: *mut PyObject
        + new(ptr: *mut PyObject) FragmentSerializer
        + serialize<S: Serializer>(serializer: S) -> Result<S::Ok, S::Error>
    }

    FragmentSerializer ..> "serde::Serialize" : implements
    FragmentSerializer ..> "pyo3_ffi::PyObject" : wraps raw pointer
    FragmentSerializer ..> "Fragment" : casts ptr for access
    FragmentSerializer ..> "PyStr" : uses for string conversion
    FragmentSerializer ..> "SerializeError" : returns on errors

Summary

The [fragment.rs](/projects/287/67788) file encapsulates the logic for converting Python fragment objects into a byte-oriented serialized form within Rust, leveraging unsafe FFI calls for direct data access and safe Rust abstractions for error handling and type checking. It plays a crucial role in bridging Python's internal data representations with Rust's serialization frameworks, enabling efficient data interchange in hybrid Python-Rust systems.