obtype.rs

Overview

The `obtype.rs` file defines functionality to classify Python objects into a well-defined internal enumeration of object types (`ObType`). It provides efficient runtime type inspection and categorization of Python objects interfaced via the Python C API (through `pyo3_ffi`). This classification is critical for serialization and processing workflows where behavior depends on the Python object type.

The core of the file is the `ObType` enum representing various Python data types and the functions `pyobject_to_obtype` and `pyobject_to_obtype_unlikely` which determine the appropriate `ObType` for a given Python object pointer (`*mut PyObject`).

Detailed Documentation

Enum: ObType

#[repr(u32)]
pub(crate) enum ObType {
    Str,
    Int,
    Bool,
    None,
    Float,
    List,
    Dict,
    Datetime,
    Date,
    Time,
    Tuple,
    Uuid,
    Dataclass,
    NumpyScalar,
    NumpyArray,
    Enum,
    StrSubclass,
    Fragment,
    Unknown,
}

• Purpose: Represents an internal classification of Python object types, optimized for serialization and type-dependent logic.

• Variants:

  • Common Python primitive types and built-in containers: Str, Int, Bool, None, Float, List, Dict, Tuple.

  • Date/time related types: Datetime, Date, Time.

  • Specialized or extended types: Uuid, Dataclass, NumpyScalar, NumpyArray, Enum.

  • Subclassed string type: StrSubclass.

  • Application-specific or domain types: Fragment.

  • Fallback when type is not recognized: Unknown.

• Representation: #[repr(u32)] for efficient storage as a 32-bit unsigned integer.

Function: pyobject_to_obtype

pub(crate) fn pyobject_to_obtype(obj: *mut pyo3_ffi::PyObject, opts: Opt) -> ObType

• Purpose: Determines and returns the ObType variant that corresponds to the given raw Python object pointer.

• Parameters:

  • obj: *mut pyo3_ffi::PyObject — Raw pointer to a Python object.

  • opts: Opt — Options bitflags controlling certain passthrough or serialization behaviors.

• Returns: ObType — The detected internal type classification.

• Behavior:

  • Uses macro ob_type!(obj) to get the Python object's type pointer.

  • Quickly checks common built-in types (str, int, bool, None, float, list, dict, datetime) using macros like is_class_by_type!(ob_type, STR_TYPE) for efficient matching.

  • If none of the common types match or certain options are enabled, delegates to pyobject_to_obtype_unlikely for extended or less common types.

• Usage Example:

let py_obj: *mut pyo3_ffi::PyObject = /* obtained from Python */;
let opts = Opt::default();
let obj_type = pyobject_to_obtype(py_obj, opts);
match obj_type {
    ObType::Int => println!("Object is an integer"),
    ObType::Str => println!("Object is a string"),
    _ => println!("Object type is other"),
}

Function: pyobject_to_obtype_unlikely

#[cfg_attr(feature = "optimize", optimize(size))]
#[inline(never)]
pub(crate) fn pyobject_to_obtype_unlikely(
    ob_type: *mut pyo3_ffi::PyTypeObject,
    opts: Opt,
) -> ObType

• Purpose: Handles the classification of less common or more complex Python types that are not covered in the fast path of pyobject_to_obtype.

• Parameters:

  • ob_type: *mut pyo3_ffi::PyTypeObject — Raw pointer to Python type object.

  • opts: Opt — Options flags influencing type detection.

• Returns: ObType — The resolved internal type.

• Behavior:

  • Checks for UUID, tuple, and domain-specific Fragment types.

  • When PASSTHROUGH_DATETIME is disabled, detects Date and Time.

  • Checks Python type flags (tp_flags) for subclass detection for string, int, list, and dict subclasses if PASSTHROUGH_SUBCLASS is disabled.

  • Detects if the type is an enumeration (Enum).

  • Identifies if the type is a dataclass based on presence of a special dataclass fields string key.

  • Supports NumPy types (NumpyScalar and NumpyArray) based on serialization options.

  • Defaults to Unknown if no matching type is found.

• Important implementation detail:

  • Marked with #[inline(never)] and size optimization attribute to reduce binary size and keep slow path out of the hot path.

• Usage: Called internally by pyobject_to_obtype when the fast path fails.

Implementation Details and Algorithms

• Macro usage: The file relies heavily on macros such as is_class_by_type!, opt_disabled!, tp_flags!, and pydict_contains! to encapsulate checks on Python types and options. This abstraction improves readability and maintainability.

• Fast path vs slow path: The design splits type detection into a fast path (pyobject_to_obtype) for common types and a slow but comprehensive path (pyobject_to_obtype_unlikely) for rarer types.

• Option flags: The Opt struct controls whether certain types are passed through or specially handled, e.g., passthrough of datetime objects or subclasses.

• Type flags and subclass detection: Uses Python C API type flags (tp_flags) to determine if a type is a subclass of built-in types.

• Dataclass detection: Checks if the Python type dictionary contains a special key (__dataclass_fields__), a heuristic to identify dataclasses.

• NumPy support: Conditional support for NumPy arrays and scalars based on serialization options.

Interaction with Other Parts of the System

• Crate modules involved:

  • opt — Provides option flags that customize type detection behavior.

  • serialize::per_type — Provides functions to recognize NumPy types.

  • typeref — Holds references to Python type objects and special strings used for type comparisons.

• Role in serialization: This file plays a foundational role in the serialization module by classifying Python objects before serializing them differently based on their type.

• Integration with Python via pyo3: Uses raw Python C API pointers (PyObject, PyTypeObject) via the pyo3_ffi to perform low-level type inspection without the overhead of higher-level wrappers.

• Extensibility: The modular design with option flags and extensible ObType enum allows easy addition of new type categories or behavior toggles.

Visual Diagram: Structure of obtype.rs

classDiagram
    class ObType {
        <<enum>>
        +Str
        +Int
        +Bool
        +None
        +Float
        +List
        +Dict
        +Datetime
        +Date
        +Time
        +Tuple
        +Uuid
        +Dataclass
        +NumpyScalar
        +NumpyArray
        +Enum
        +StrSubclass
        +Fragment
        +Unknown
    }

    class pyobject_to_obtype {
        +obj: *mut PyObject
        +opts: Opt
        +-> ObType
    }

    class pyobject_to_obtype_unlikely {
        +ob_type: *mut PyTypeObject
        +opts: Opt
        +-> ObType
    }

    pyobject_to_obtype --> ObType : returns
    pyobject_to_obtype_unlikely --> ObType : returns
    pyobject_to_obtype "1" --> "1" pyobject_to_obtype_unlikely : calls on miss

Summary

The `obtype.rs` file is a utility module that classifies Python objects into an internal `ObType` enum by analyzing Python type pointers and options flags. It supports both common and specialized Python types, including subclass detection, dataclasses, enums, and NumPy types. Critical to serialization and runtime type-dependent logic, this module balances performance via a fast path and extensibility with a comprehensive slow path. It interfaces closely with Python internals through the `pyo3_ffi` bindings and coordinates with other modules handling options and serialization specifics.