durable.rs

Overview

This file defines the DurableStore enum, which serves as an abstraction layer over two types of persistent key-value stores: an in-memory store (MemStore) and an Aerospike-based distributed store (AerospikeStore). It provides a unified interface to perform common database operations like get, put, and batch_get, delegating the actual work to the underlying store implementations. This abstraction allows the rest of the system to switch between storage backends seamlessly without changing the code that interacts with the storage.

The file also implements the KeyValueStore trait for DurableStore, ensuring that the enum conforms to the expected interface for key-value operations.

Components

Enum: DurableStore

• Variants:

  • Mem(MemStore): Represents an in-memory key-value store, useful primarily for testing or scenarios requiring fast, ephemeral storage.

  • Aerospike(SplitValueStore): Represents a persistent distributed key-value store backed by Aerospike, with data splitting capabilities for handling large values.

• Purpose: Encapsulates different storage backends and exposes a consistent API for key-value storage.

Methods on DurableStore

pub fn mem() -> DurableStore

• Description: Factory method to create a DurableStore instance backed by an in-memory store.

• Parameters: None

• Returns: A DurableStore::Mem variant wrapping a newly initialized MemStore.

• Usage Example:

  let store = DurableStore::mem();
  

pub fn aerospike(socket_address: String, metrics: Option<BlockProductionMetrics>) -> anyhow::Result<Self>

• Description: Factory method to create a DurableStore instance backed by an Aerospike store.

• Parameters:

  • socket_address (String): The network address of the Aerospike server.

  • metrics (Option): Optional metrics collector for block production.

• Returns: Ok(DurableStore::Aerospike(_)) on success or an error wrapped in anyhow::Result if initialization fails.

• Details: Internally, it creates an AerospikeStore instance and then wraps it with SplitValueStore configured with a chunk size of approximately 1MB (1024 * 1020 bytes).

• Usage Example:

  let store = DurableStore::aerospike("127.0.0.1:3000".to_string(), None)?;
  

Trait Implementation: KeyValueStore for DurableStore

Implements the KeyValueStore trait to provide key-value storage operations by delegating to the underlying store variant.

Functions

fn get(&self, key: &Key, bins: &[&str], label: &'static str) -> anyhow::Result<Option<BinMap>>

• Description: Retrieves data associated with a given key and requested bins.

• Parameters:

  • key (&Key): The key to lookup.

  • bins (&[&str]): A list of bin names to fetch.

  • label (&'static str): A static label for logging or metrics.

• Returns: A Result wrapping an optional BinMap (mapping of bin names to values).

• Behavior: Calls the corresponding get method on the underlying store.

• Usage Example:

  let result = store.get(&some_key, &["bin1", "bin2"], "read_label")?;
  

fn put(&self, key: &Key, bins: &[Bin], until: bool, label: &'static str) -> anyhow::Result<()>

• Description: Stores or updates bins associated with a given key.

• Parameters:

  • key (&Key): The key under which to store data.

  • bins (&[Bin]): Bins containing the data to write.

  • until (bool): A flag indicating some conditional behavior on write (context-dependent).

  • label (&'static str): A static label for logging or metrics.

• Returns: A Result indicating success or failure.

• Behavior: Delegates the put call to the underlying store.

• Usage Example:

  store.put(&some_key, &bins, false, "write_label")?;
  

fn batch_get(&self, reads: Vec<BatchGet>) -> anyhow::Result<Vec<Option<BinMap>>>

• Description: Performs batch retrieval of multiple keys and bins.

• Parameters:

  • reads (Vec<BatchGet>): A vector of batch get requests.

• Returns: A vector of optional BinMaps wrapped in a Result.

• Behavior: Delegates batch reads to the underlying store.

• Usage Example:

  let batch_reads = vec![BatchGet::new(...), BatchGet::new(...)];
  let results = store.batch_get(batch_reads)?;
  

Debug-only Functions (enabled under debug_assertions)

• fn db_reads(&self) -> usize: Returns the number of database read operations performed.

• fn db_writes(&self) -> usize: Returns the number of database write operations performed.

These methods are useful for monitoring and debugging database access patterns and are delegated to the underlying store.

Implementation Details and Algorithms

• The DurableStore uses an enum to abstract two distinct storage implementations, leveraging Rust's powerful pattern matching to route method calls.

• The Aerospike variant uses a SplitValueStore wrapper which likely implements chunking or splitting of large values to comply with Aerospike record size limits (chunk size set around 1MB).

• The put and get operations include a label parameter, which can be used for logging or metrics collection, aiding observability.

• Conditional compilation (#[cfg(debug_assertions)]) is used to include debug-specific methods for tracking database operations only in debug builds.

Interactions with Other Modules

• Depends on external Aerospike client types (aerospike::Key, aerospike::Bin).

• Uses internal modules:

  • MemStore from crate::storage::mem as the in-memory store implementation.

  • AerospikeStore from crate::storage as the Aerospike backend.

  • SplitValueStore to handle splitting large values for Aerospike.

  • BatchGet, BinMap, and KeyValueStore traits from crate::storage.

  • BlockProductionMetrics from crate::helper::metrics for optional metrics integration.

• The abstraction enables seamless switching between memory and Aerospike storage without modifying client code.

Diagram: DurableStore Structure and Method Delegation

classDiagram
class DurableStore {
+mem()
+aerospike()
+get()
+put()
+batch_get()
+db_reads()
+db_writes()
}
class MemStore {
+get()
+put()
+batch_get()
+db_reads()
+db_writes()
}
class AerospikeStore {
+new()
+get()
+put()
+batch_get()
+db_reads()
+db_writes()
}
class SplitValueStore {
+new()
+get()
+put()
+batch_get()
+db_reads()
+db_writes()
}
DurableStore --> MemStore : Mem variant
DurableStore --> SplitValueStore : Aerospike variant
SplitValueStore --> AerospikeStore : wraps

This diagram illustrates the DurableStore enum holding either a MemStore or a SplitValueStore wrapping an AerospikeStore. The methods on DurableStore delegate calls to the respective methods on the contained store.