pyobject.rs

Overview

The [pyobject.rs](/projects/287/67800) file provides utility functions to facilitate the creation and management of Python objects within a Rust environment, specifically targeting interoperability with Python's C API via `pyo3` and related FFI (Foreign Function Interface) constructs. The primary focus of this module is the efficient conversion of Rust primitive types and strings into Python objects (`PyObject` pointers), leveraging internal caching mechanisms and optimizations to enhance performance, especially for commonly used immutable Python objects such as `True`, `False`, and `None`.

Key features include:

• Conversion of Rust primitive types (bool, i64, u64, f64) into Python objects.

• Efficient string key caching for Python Unicode string objects to reduce overhead related to string interning.

• Use of compile-time conditional compilation to handle situations where the Python Global Interpreter Lock (GIL) is disabled.

• Safe encapsulation of raw pointer usage to Python objects with NonNull pointers to ensure non-null guarantees.

This file is typically used internally in a larger system that binds Rust and Python together, enabling seamless data exchange and Python object manipulation within Rust code.

Detailed Explanation of Functions

get_unicode_key(key_str: &str) -> PyStr

Returns a cached or newly created Python Unicode string object (`PyStr`) corresponding to the provided Rust string slice `key_str`.

• Parameters:

  • key_str: &str — The Rust string slice to be converted or cached as a Python Unicode string.

• Returns:

  • PyStr — A Python Unicode string wrapper.

• Behavior:

  • When the Python Global Interpreter Lock (GIL) is enabled:

    • If the string length exceeds 64 characters, a new PyStr is created with its hash computed.

    • Otherwise, a hash of the string is computed using xxh3_64. This hash is used as a key into a global cache (KEY_MAP) where an existing PyStr may be retrieved or stored.

    • This caching avoids repeated allocations or hash computations for frequently used string keys.

  • When the GIL is disabled, it simply returns a new PyStr with the hash computed without caching.

• Usage Example:

  let key = "example_key";
  let py_str = get_unicode_key(key);
  // py_str can now be used where a Python Unicode string object is required.
  

parse_bool(val: bool) -> NonNull<pyo3_ffi::PyObject>

Converts a Rust boolean value into a Python `True` or `False` object pointer.

• Parameters:

  • val: bool — The boolean value to convert.

• Returns:

  • NonNull<pyo3_ffi::PyObject> — A non-null pointer to the Python boolean object.

• Behavior:

  • Returns a pointer to the singleton Python True object if val is true.

  • Returns a pointer to the singleton Python False object if val is false.

• Usage Example:

  let py_true = parse_bool(true);
  let py_false = parse_bool(false);
  

parse_true() -> NonNull<pyo3_ffi::PyObject>

Returns a pointer to the Python singleton `True` object.

• Returns:

  • NonNull<pyo3_ffi::PyObject> — Pointer to Python True.

• Usage Example:

  let py_true = parse_true();
  

parse_false() -> NonNull<pyo3_ffi::PyObject>

Returns a pointer to the Python singleton `False` object.

• Returns:

  • NonNull<pyo3_ffi::PyObject> — Pointer to Python False.

• Usage Example:

  let py_false = parse_false();
  

parse_i64(val: i64) -> NonNull<pyo3_ffi::PyObject>

Converts a Rust signed 64-bit integer into a Python integer object.

• Parameters:

  • val: i64 — The integer value to convert.

• Returns:

  • NonNull<pyo3_ffi::PyObject> — Pointer to Python integer object.

• Behavior:

  • Calls PyLong_FromLongLong via FFI to create a Python integer object from the Rust i64.

• Usage Example:

  let py_int = parse_i64(42);
  

parse_u64(val: u64) -> NonNull<pyo3_ffi::PyObject>

Converts a Rust unsigned 64-bit integer into a Python integer object.

• Parameters:

  • val: u64 — The unsigned integer value to convert.

• Returns:

  • NonNull<pyo3_ffi::PyObject> — Pointer to Python integer object.

• Behavior:

  • Calls PyLong_FromUnsignedLongLong via FFI to create a Python integer object from the Rust u64.

• Usage Example:

  let py_uint = parse_u64(42);
  

parse_f64(val: f64) -> NonNull<pyo3_ffi::PyObject>

Converts a Rust 64-bit floating-point number into a Python float object.

• Parameters:

  • val: f64 — The floating-point value to convert.

• Returns:

  • NonNull<pyo3_ffi::PyObject> — Pointer to Python float object.

• Behavior:

  • Calls PyFloat_FromDouble via FFI to create a Python float object from the Rust f64.

• Usage Example:

  let py_float = parse_f64(3.14);
  

parse_none() -> NonNull<pyo3_ffi::PyObject>

Returns a pointer to the Python singleton `None` object.

• Returns:

  • NonNull<pyo3_ffi::PyObject> — Pointer to Python None.

• Usage Example:

  let py_none = parse_none();
  

Implementation Details and Algorithms

• String Caching with get_unicode_key:

  • Uses the xxh3_64 hash function (from the xxhash_rust crate) to hash string keys efficiently.

  • Employs a caching mechanism (KEY_MAP) that maps hashes to cached Python string objects (CachedKey), avoiding repeated Python string creation for common keys.

  • The cache is only enabled when GIL is active to ensure thread safety.

  • For long strings (>64 characters), it bypasses caching to avoid overhead.

  • Uses unsafe Rust code (unsafe blocks) and unwrap_or_else to optimize performance while assuming correctness guarantees from the runtime environment.

• Singleton Python Objects:

  • The objects TRUE, FALSE, and NONE represent singleton Python objects that are immortal and shared.

  • The macros nonnull! and use_immortal! ensure safe usage and lifetime guarantees for these objects without unnecessary reference counting overhead.

• Foreign Function Interface (FFI):

  • The functions parse_i64, parse_u64, and parse_f64 call the appropriate Python C API functions to create Python numeric objects from Rust primitives.

  • The returned pointers are wrapped in NonNull to guarantee non-nullability, enhancing safety.

• Compile-time Feature Flags:

  • Conditional compilation attributes control behavior depending on whether Py_GIL_DISABLED is enabled.

  • This allows the code to adapt to environments where the Python GIL might be disabled, affecting thread-safety and caching strategies.

Interaction with Other Parts of the System

• Integration with Python FFI Layer:

  • This module works closely with the Python FFI layer (pyo3_ffi) to convert Rust data types into Python objects.

  • It depends on PyStr and CachedKey from other internal modules (deserialize::cache, str) for string handling and caching.

• Global Cache (KEY_MAP):

  • The global cache for string keys is shared across the system and is critical for performance-sensitive operations involving Python string keys (e.g., attribute lookups, dictionary keys).

• Immortal Objects (TRUE, FALSE, NONE):

  • These are defined elsewhere as static references to singleton Python objects, shared throughout the system to avoid repeated creation.

• Macros and Unsafe Code:

  • The file uses macros such as nonnull!, use_immortal!, and ffi! which are part of the overall project's internal utilities to safely wrap raw pointers and FFI calls.

Visual Diagram

classDiagram
    class pyobject {
        +get_unicode_key(key_str: &str) PyStr
        +parse_bool(val: bool) NonNull<PyObject>
        +parse_true() NonNull<PyObject>
        +parse_false() NonNull<PyObject>
        +parse_i64(val: i64) NonNull<PyObject>
        +parse_u64(val: u64) NonNull<PyObject>
        +parse_f64(val: f64) NonNull<PyObject>
        +parse_none() NonNull<PyObject>
    }
    class PyStr {
        +from_str_with_hash()
    }
    class CachedKey {
        +new(PyStr)
        +get()
    }
    pyobject ..> PyStr : uses
    pyobject ..> CachedKey : uses

Summary

The [pyobject.rs](/projects/287/67800) file provides foundational utilities for converting Rust primitives and strings into Python objects with performance-conscious caching and safe pointer handling. It abstracts away low-level FFI details and ensures that commonly used Python singleton objects are efficiently accessed. This module is a crucial component in Rust-Python interoperability within the system and supports higher-level Python bindings and serialization/deserialization tasks.