mod.rs

Overview

This file defines the CrossRefStorage struct, which provides an abstraction layer for storing and retrieving serialized data blobs using a key-value storage backend. It is designed to work with various implementations of the KeyValueStore trait, allowing flexible storage options including in-memory and Aerospike-based persistence.

The primary responsibility of CrossRefStorage is to manage data blobs identified by a string key (referred to as path in methods) and to serialize/deserialize Rust types transparently using bincode. This facilitates efficient cross-referencing and storage of arbitrary Rust data structures in a binary format.

Structures and Functions

CrossRefStorage

A cloneable struct encapsulating a thread-safe shared reference (Arc) to a key-value store and a string prefix for Aerospike set naming.

Fields

• store: Arc<dyn KeyValueStore>
  A thread-safe reference to the underlying key-value storage backend. The trait object allows plugging in different storage implementations.

• set_prefix: String
  A prefix string to customize the Aerospike set name under which keys are stored. This helps namespace segregation.

Methods

new(store: impl KeyValueStore + 'static, set_prefix: &str) -> Self

Creates a new CrossRefStorage instance wrapping the provided key-value store implementation and set prefix.

• Parameters:

  • store: Any object implementing KeyValueStore trait.

  • set_prefix: A string slice used as a prefix for Aerospike set names.

• Returns:
  A new CrossRefStorage instance.

• Usage Example:

  let mem_store = MemStore::new();
  let storage = CrossRefStorage::new(mem_store, "my_prefix");
  

mem() -> CrossRefStorage

Convenience method to create a CrossRefStorage backed by an in-memory store (MemStore) with an empty set prefix.

• Returns:
  A CrossRefStorage instance using in-memory storage.

• Usage Example:

  let mem_storage = CrossRefStorage::mem();
  

message_key(&self, hash: &str) -> Key

Generates a key for the storage backend by combining the configured namespace (NAMESPACE constant), the set prefix, and the provided hash string.

• Parameters:

  • hash: The string key that identifies the data blob.

• Returns:
  An Aerospike Key object representing the composite key.

• Implementation Details:
  This method uses the as_key! macro from the Aerospike client library, which constructs keys in the format (namespace, set, user_key).

read_blob<T: for<'de> Deserialize<'de>>(&self, path: &str) -> anyhow::Result<Option<T>>

Reads and deserializes a data blob stored at the given path.

• Parameters:

  • path: The string key identifying the stored blob.

• Returns:
  A Result containing an optional deserialized object of type T if found; otherwise None.

• Error Handling:
  Returns an error if the record is missing the expected blob bin or if deserialization fails.

• Usage Example:

  let result: Option<MyStruct> = storage.read_blob("my_key")?;
  if let Some(data) = result {
      // use data
  }
  

• Implementation Details:

  • Calls store.get with the generated key, requesting the BIN_BLOB bin.

  • Validates the presence of the blob bin and extracts its binary value.

  • Uses bincode::deserialize to convert the binary back to the desired Rust type.

write_blob<T: Serialize>(&self, path: &str, data: T, until_success: bool) -> anyhow::Result<()>

Serializes and writes the provided data blob into the storage backend under the specified path.

• Parameters:

  • path: The string key where the blob will be stored.

  • data: The data structure to serialize and store.

  • until_success: A boolean flag indicating whether the write operation should retry until successful.

• Returns:
  A Result indicating success or failure of the write operation.

• Usage Example:

  storage.write_blob("my_key", &my_data, true)?;
  

• Implementation Details:

  • Serializes the data using bincode::serialize.

  • Wraps the serialized bytes in an Aerospike bin using as_bin!.

  • Calls store.put with the generated key, bins, and retry flag.

Implementation Details and Algorithms

• The storage keys are generated combining a fixed namespace (NAMESPACE), a customizable set prefix, and the user-provided hash (or path). This design supports multi-tenancy or logical segregation within the same Aerospike namespace.

• Data is stored and retrieved in binary form in a single bin named by the constant BIN_BLOB. This bin is expected to contain the serialized representation of arbitrary Rust types.

• Serialization and deserialization use the bincode crate, which provides compact binary encoding for Rust data structures. This choice optimizes storage and transmission efficiency.

• The storage backend is abstracted by the KeyValueStore trait, allowing different implementations such as in-memory (MemStore) or Aerospike-based persistence to be swapped transparently.

• Error handling is performed using the anyhow crate, enabling convenient error propagation with context.

Interaction with Other System Components

• KeyValueStore Trait
  The core storage functionality depends on the KeyValueStore trait from the crate::storage module. This trait defines get and put operations used by CrossRefStorage.

• MemStore
  The mem() method creates an instance backed by MemStore, an in-memory key-value store useful for testing or ephemeral data scenarios.

• Aerospike Client Macros and Types
  The file imports Aerospike macros (as_key!, as_bin!) and types (Key, Value) for constructing keys and bins compatible with Aerospike's data model.

• Metrics
  The constant AEROSPIKE_OBJECT_TYPE_CROSS_REF_DATA is passed to storage operations to tag metrics or telemetry related to cross-reference data storage.

• Constants
  BIN_BLOB and NAMESPACE are constants from the crate::storage module defining the bin name and Aerospike namespace respectively.

Visual Diagram

classDiagram
class CrossRefStorage {
-store: Arc<KeyValueStore>
-set_prefix: String
+new()
+mem()
-message_key()
+read_blob()
+write_blob()
}
CrossRefStorage ..> KeyValueStore : uses
CrossRefStorage ..> MemStore : can use
CrossRefStorage ..> Key : constructs keys
CrossRefStorage ..> Value : reads/writes bins