unprocessed_blocks_collection.rs

Overview

This file implements a thread-safe collection and management system for unfinalized candidate blocks within a blockchain or distributed ledger context. The primary purpose is to maintain blocks that have not yet been finalized, allowing insertion, filtering, retrieval, and removal based on their state and sequence numbers. It supports notification mechanisms to alert subscribers about changes in the collection.

Key responsibilities include:

• Storing blocks with their associated state and cryptographic envelopes.

• Filtering blocks based on sequence numbers to prune outdated or irrelevant entries.

• Providing thread-safe access and mutation operations guarded by synchronization primitives.

• Managing notifications linked to block state changes.

Main Structures and Their Functionality

UnfinalizedBlocksSnapshot

pub struct UnfinalizedBlocksSnapshot {
    blocks: BTreeMap<BlockIndex, (BlockState, Arc<Envelope<GoshBLS, AckiNackiBlock>>)>,
    notifications_stamp: u32,
    block_id_set: HashSet<BlockIdentifier>,
}

• Purpose: Immutable snapshot representation of the current unfinalized blocks collection.

• Properties:

  • blocks: Sorted map of blocks by their BlockIndex containing both the block state and the cryptographic envelope.

  • notifications_stamp: A version or timestamp indicating when notifications were last updated.

  • block_id_set: A set of block identifiers for quick membership checks.

• Traits: Implements Default and Getters for field access. Also implements AllowGuardedMut to allow guarded mutable access elsewhere.

UnfinalizedBlocksData

struct UnfinalizedBlocksData {
    main_map: UnfinalizedBlocksSnapshot,
    identifier_to_seqno: HashMap<BlockIdentifier, BlockSeqNo>,
    filter: FilterPrehistoric,
}

• Purpose: Internal mutable data structure holding the blocks collection and auxiliary indexing.

• Properties:

  • main_map: The UnfinalizedBlocksSnapshot containing the main blocks data.

  • identifier_to_seqno: Maps block identifiers to their sequence numbers for efficient lookup.

  • filter: A FilterPrehistoric instance used to filter out blocks older than a certain sequence number.

• Debugging: Uses Dbg derive macro, skipping the main_map to avoid verbose output.

Methods

• set_filter(&mut self, filter: FilterPrehistoric): Updates the filter and retains only blocks with sequence numbers greater than the filter's.

• insert(&mut self, index: BlockIndex, value: (BlockState, Arc<Envelope<GoshBLS, AckiNackiBlock>>)): Adds a new block to the collection and updates the auxiliary index.

• retain<F>(&mut self, f: F): Retains blocks in the collection based on a predicate function f. Removes blocks and cleans up auxiliary mappings if they don't satisfy f.

FilterPrehistoric

pub struct FilterPrehistoric {
    block_seq_no: BlockSeqNo,
}

• Purpose: Encapsulates a filter criterion to exclude blocks with sequence numbers older than block_seq_no.

• Traits: Implements Default, Clone, PartialEq, Eq, Getters, TypedBuilder, and Dbg.

• Usage: Used by UnfinalizedBlocksData to prune blocks and in collection filtering operations.

UnfinalizedCandidateBlockCollection

pub struct UnfinalizedCandidateBlockCollection {
    candidates: Arc<Mutex<UnfinalizedBlocksData>>,
    notifications: Notification,
}

• Purpose: Thread-safe public interface for managing unfinalized candidate blocks.

• Properties:

  • candidates: Shared, mutex-protected UnfinalizedBlocksData for concurrent access.

  • notifications: Notification system that signals subscribers on collection changes.

• Traits: Implements Dbg (skips notifications), Clone, and AllowGuardedMut.

Constructor

• new(states: impl Iterator<Item = (BlockState, Arc<Envelope<GoshBLS, AckiNackiBlock>>>) ) -> Self
  Initializes the collection from an iterator of block states and envelopes. Inserts all provided blocks into the internal data structure and creates a new notification instance.

Key Methods

• get_block_by_id(&self, identifier: &BlockIdentifier) -> Option<Arc<Envelope<GoshBLS, AckiNackiBlock>>>
  Retrieves a block envelope by its identifier if present.

• clone_queue(&self) -> (UnfinalizedBlocksSnapshot, FilterPrehistoric)
  Returns a snapshot of the current blocks and the active filter, along with the notification stamp.

• insert(&mut self, block_state: BlockState, block: Envelope<GoshBLS, AckiNackiBlock>)
  Inserts a new block and its state into the collection, adds a subscriber for notifications to the block state, and triggers a notification update.

• remove_finalized_and_invalidated_blocks(&mut self)
  Removes blocks that are either finalized or invalidated from the collection.

• remove_old_blocks(&mut self, last_finalized_block_seq_no: &BlockSeqNo)
  Removes blocks older than a provided last finalized block sequence number.

• retain<F>(&mut self, action: F)
  Retains blocks in the collection based on a predicate function applied to their states. If blocks are removed, subscribers are notified accordingly.

• notifications(&self) -> &Notification
  Returns a reference to the notifications object for subscribing or polling changes.

• touch(&mut self)
  Manually triggers an update to the notification stamp, signaling subscribers.

• update_filter(&self, filter_prehistoric: FilterPrehistoric) -> bool
  Updates the filter if the new filter's sequence number is greater than the current one. Returns true if updated, otherwise false.

Important Implementation Details and Algorithms

• The collection uses a BTreeMap keyed by BlockIndex to maintain blocks in sequence order, which facilitates range queries and ordered traversal.

• Auxiliary hash maps (identifier_to_seqno and block_id_set) provide quick lookups by block identifier and membership checks.

• The filtering mechanism in UnfinalizedBlocksData leverages sequence numbers to efficiently prune blocks older than a given threshold.

• The use of Arc<Envelope<GoshBLS, AckiNackiBlock>> for blocks ensures the blocks are reference-counted and shared safely across threads.

• Synchronization is handled via parking_lot::Mutex wrapped in an Arc for safe concurrent access and mutation.

• The Notification object supports a publish-subscribe pattern to notify interested parties of changes, ensuring reactive updates.

• The retain method in UnfinalizedCandidateBlockCollection performs selective removal with explicit tracking of removed blocks to update subscribers accordingly.

• Guards (AllowGuardedMut, Guarded, GuardedMut) are used extensively to enforce safe mutable access patterns, which is crucial in concurrent environments.

Interactions with Other Parts of the System

• BlockState: Represents the state of a block and supports subscriber management for notifications.

• Envelope: Wraps blocks with cryptographic signatures or proofs, specifically using GoshBLS schemes.

• BlockIndex, BlockIdentifier, BlockSeqNo: Types that represent block metadata crucial for ordering and identification.

• Notification: Provides a mechanism to signal changes to observers, enabling reactive system components.

• The collection is likely used by consensus or block processing modules to handle candidate blocks before they become finalized.

Usage Examples

Creating a New Collection from Existing States

let states_iter = some_iterator_of_blocks(); // Iterator<Item = (BlockState, Arc<Envelope<GoshBLS, AckiNackiBlock>>)>
let collection = UnfinalizedCandidateBlockCollection::new(states_iter);

Inserting a New Block

collection.insert(block_state, block_envelope);

Retrieving a Block by Identifier

if let Some(block) = collection.get_block_by_id(&block_identifier) {
    // Use block envelope
}

Removing Finalized and Invalidated Blocks

collection.remove_finalized_and_invalidated_blocks();

Updating the Filter to Remove Old Blocks

let filter = FilterPrehistoric::builder()
    .block_seq_no(new_seq_no)
    .build();
collection.update_filter(filter);

Mermaid Diagram: Class Structure and Relationships

classDiagram
class UnfinalizedCandidateBlockCollection {
+new()
+get_block_by_id()
+clone_queue()
+insert()
+remove_finalized_and_invalidated_blocks()
+remove_old_blocks()
+retain()
+notifications()
+touch()
+update_filter()
}
class UnfinalizedBlocksData {
-main_map: UnfinalizedBlocksSnapshot
-identifier_to_seqno: HashMap
-filter: FilterPrehistoric
+set_filter()
+insert()
+retain()
}
class UnfinalizedBlocksSnapshot {
-blocks: BTreeMap
-notifications_stamp: u32
-block_id_set: HashSet
}
class FilterPrehistoric {
-block_seq_no: BlockSeqNo
}
UnfinalizedCandidateBlockCollection --> UnfinalizedBlocksData : manages >
UnfinalizedBlocksData --> UnfinalizedBlocksSnapshot : contains >
UnfinalizedBlocksData --> FilterPrehistoric : uses >

This diagram illustrates the containment and management hierarchy among the core structures, emphasizing the thread-safe management of unfinalized blocks and filtering behavior.