long.rs

Overview

The `long.rs` file provides low-level Rust bindings and utilities to interact with Python's internal representation of long integers (`PyLongObject`) within the PyO3 FFI layer. It primarily focuses on supporting Python 3.12's new long integer internal layout while maintaining backward compatibility with earlier Python versions.

This file defines internal Rust representations of Python’s long integer structures and includes several inline helper functions to query and manipulate these objects efficiently. These helpers facilitate checking the sign of the long integer, determining if it fits within a 32-bit signed integer, checking if the integer is zero, and extracting the integer's inline value. The implementation leverages conditional compilation to adapt to changes introduced in Python 3.12 and optional features like `"inline_int"`.

Detailed Description

Structs

_PyLongValue

#[repr(C)]
pub(crate) struct _PyLongValue {
    pub lv_tag: usize,
    pub ob_digit: u32,
}

• Purpose: Represents the internal tagged value of a Python long integer in Python 3.12.

• Fields:

  • lv_tag: A usize that holds tag bits (including sign information).

  • ob_digit: The actual digit representing the integer value (unsigned 32-bit).

• Usage: Used inside PyLongObject to store the tagged integer value with embedded sign and magnitude.

PyLongObject

There are two variants depending on the Python version:

• For Python 3.12 and later:

#[repr(C)]
pub(crate) struct PyLongObject {
    pub ob_base: pyo3_ffi::PyObject,
    pub long_value: _PyLongValue,
}

• For older Python versions (< 3.12):

#[repr(C)]
pub(crate) struct PyLongObject {
    pub ob_base: pyo3_ffi::PyVarObject,
    pub ob_digit: u32,
}

• Purpose: Represents the Python int object at the C level.

• Fields:

  • ob_base: The base Python object (either PyObject or PyVarObject depending on version).

  • long_value or ob_digit: The integer digits or tagged value as explained above.

Constants

Defined only when targeting Python 3.12 and optionally with `"inline_int"` feature:

• SIGN_MASK = 3: Mask to extract the sign bits from lv_tag.

• SIGN_ZERO = 1: Represents the zero value tag.

• NON_SIZE_BITS = 3: Number of bits used for non-size information in the tag.

These constants are crucial for decoding the tagged integer representation introduced in Python 3.12.

Functions

All functions are marked `pub(crate)` and `inline(always)` for efficient FFI calls.

pylong_is_unsigned

pub(crate) fn pylong_is_unsigned(ptr: *mut pyo3_ffi::PyObject) -> bool

• Parameters:

  • ptr: A raw pointer to a Python object expected to be a PyLongObject.

• Returns: true if the Python integer is unsigned (non-negative), false otherwise.

• Behavior:

  • For Python 3.12: Checks if the sign bits in lv_tag are zero.

  • For older Python: Checks if ob_size > 0 (where ob_size is the number of digits with sign).

• Usage Example:

if pylong_is_unsigned(py_long_ptr) {
    println!("The Python long integer is non-negative.");
} else {
    println!("The Python long integer is negative.");
}

pylong_fits_in_i32

pub(crate) fn pylong_fits_in_i32(ptr: *mut pyo3_ffi::PyObject) -> bool

• Parameters:

  • ptr: Raw pointer to a Python long object.

• Returns: true if the integer fits within a signed 32-bit integer, false otherwise.

• Behavior:

  • Python 3.12: Checks if lv_tag is less than (2 << NON_SIZE_BITS).

  • Older Python: Checks if the absolute value of ob_size is 1 (single digit).

• Note: The logic uses the internal structure to quickly identify small integers.

• Usage Example:

if pylong_fits_in_i32(py_long_ptr) {
    println!("Integer fits in i32.");
} else {
    println!("Integer too large for i32.");
}

pylong_is_zero

pub(crate) fn pylong_is_zero(ptr: *mut pyo3_ffi::PyObject) -> bool

• Parameters:

  • ptr: Raw pointer to Python long integer object.

• Returns: true if the value is zero, else false.

• Behavior:

  • Python 3.12: Checks if lv_tag & SIGN_MASK equals SIGN_ZERO.

  • Older Python: Checks if ob_size == 0.

• Usage Example:

if pylong_is_zero(py_long_ptr) {
    println!("Integer is zero.");
} else {
    println!("Integer is non-zero.");
}

pylong_get_inline_value

pub(crate) fn pylong_get_inline_value(ptr: *mut pyo3_ffi::PyObject) -> i64

• Parameters:

  • ptr: Raw pointer to Python long integer object.

• Returns: The integer value as a signed 64-bit integer (i64).

• Behavior:

  • Python 3.12: Returns ob_digit as positive or negated i64 based on sign bits.

  • Older Python: Multiplies ob_size (which encodes sign and digit count) by the digit value.

• Important: This function assumes the integer fits in the inline representation (checked by pylong_fits_in_i32).

• Usage Example:

let value = pylong_get_inline_value(py_long_ptr);
println!("Inline integer value: {}", value);

Implementation Details and Algorithms

• Tagged Integer Representation:
  Python 3.12 introduces a tagged integer representation to optimize small integers by packing sign and magnitude into a single field (lv_tag), reducing memory overhead and improving performance. The constants SIGN_MASK, SIGN_ZERO, and NON_SIZE_BITS are used to decode this tag.

• Conditional Compilation:
  The file uses Rust’s #[cfg] attribute to compile different code paths depending on the Python version and enabled features ("inline_int"). This ensures compatibility and optimal performance across Python versions.

• Unsafe Pointer Casting:
  All functions use unsafe Rust code to cast raw pointers (*mut PyObject) to the appropriate internal struct pointers (PyLongObject, _PyLongValue) and directly access fields, reflecting the low-level FFI nature of this module.

• Inline Functions:
  Functions are marked #[inline(always)] to minimize overhead and encourage compiler optimization since these operations are expected to be called frequently in performance-sensitive code.

Interaction with Other Parts of the System

• This file is part of the PyO3 FFI bindings that provide Rust access to Python internals.

• It interacts closely with the pyo3_ffi crate types such as PyObject and PyVarObject.

• Other modules handling Python object conversions, arithmetic, or type checking can call these helper functions to efficiently extract and analyze Python integers at the FFI boundary.

• The inline integer feature ("inline_int") must be enabled to leverage specialized handling for small integers.

• This module does not provide high-level Python integer operations but supports them by enabling fast introspection and extraction of integer values.

Visual Diagram

classDiagram
    class _PyLongValue {
        +usize lv_tag
        +u32 ob_digit
    }

    class PyLongObject_3_12 {
        +pyo3_ffi::PyObject ob_base
        +_PyLongValue long_value
    }

    class PyLongObject_pre_3_12 {
        +pyo3_ffi::PyVarObject ob_base
        +u32 ob_digit
    }

    PyLongObject_3_12 --> _PyLongValue

Summary

The `long.rs` file encapsulates the Python long integer's internal representation and provides efficient, version-dependent utilities for inspecting and extracting integer values from Python objects in Rust. It is designed for use within the PyO3 FFI ecosystem to bridge Rust and Python integer handling with attention to the optimizations introduced in Python 3.12.