send.rs

Overview

This file implements network communication functions and methods related to sending and broadcasting critical blockchain synchronization messages within a node. It primarily deals with requesting blocks, broadcasting node joining events, synchronization finalization, and acknowledgments among nodes in a distributed network. The interactions involve direct messaging to specific nodes or broadcasting messages to all nodes in the network.

The file defines standalone functions for sending specific types of messages and provides implementations of these functionalities as methods on the generic Node struct. The Node struct methods utilize various network channels (NetDirectSender and NetBroadcastSender) to communicate with other nodes. Error handling is performed carefully to handle shutdown scenarios gracefully.

Functions

send_blocks_range_request

pub fn send_blocks_range_request(
    network_direct_tx: &NetDirectSender<NodeIdentifier, NetworkMessage>,
    destination_node_id: NodeIdentifier,
    requester: NodeIdentifier,
    thread_id: ThreadIdentifier,
    inclusive_from: BlockSeqNo,
    exclusive_to: BlockSeqNo,
    at_least_n_blocks: Option<usize>,
) -> anyhow::Result<()>

Sends a direct network message to a specific node requesting a range of blocks in a given thread.

• Parameters:

  • network_direct_tx: Reference to a direct sender for network messages.

  • destination_node_id: The target node to which the request is sent.

  • requester: The node making the request (usually the local node).

  • thread_id: Identifier of the thread for which blocks are requested.

  • inclusive_from: The starting block sequence number (inclusive).

  • exclusive_to: The ending block sequence number (exclusive).

  • at_least_n_blocks: Optional minimum number of blocks required.

• Returns:
  Returns Ok(()) if the message is sent successfully or an error if sending fails (unless the system is shutting down).

• Usage Example:

send_blocks_range_request(
    &network_direct_tx,
    destination_node_id,
    requester_node_id,
    thread_id,
    100,
    200,
    Some(50),
)?;

This function logs the request details and sends a NetworkMessage::BlockRequest containing the range and requester information.

broadcast_node_joining

pub(crate) fn broadcast_node_joining(
    network_broadcast_tx: &NetBroadcastSender<NetworkMessage>,
    metrics: Option<&BlockProductionMetrics>,
    node_id: NodeIdentifier,
    thread_id: ThreadIdentifier,
) -> anyhow::Result<()>

Broadcasts a message to all nodes indicating that a new node is joining a specific thread.

• Parameters:

  • network_broadcast_tx: Reference to a broadcast sender for network messages.

  • metrics: Optional reference to metrics for reporting broadcast events.

  • node_id: The identifier of the joining node.

  • thread_id: The thread to which the node is joining.

• Returns:
  Returns Ok(()) on success or an error if the broadcast fails (except during shutdown).

• Usage Example:

broadcast_node_joining(
    &network_broadcast_tx,
    Some(&metrics),
    joining_node_id,
    thread_id,
)?;

It logs the broadcast event and reports metrics if provided.

Node Struct Methods

The methods defined on the generic Node<TStateSyncService, TRandomGenerator> struct provide convenient wrappers and additional logic for sending and broadcasting network messages related to block synchronization and node coordination.

Generic Constraints

• TStateSyncService: Must implement StateSyncService with a repository of type RepositoryImpl.

• TRandomGenerator: Must implement the rand::Rng trait.

broadcast_node_joining

pub(crate) fn broadcast_node_joining(&self) -> anyhow::Result<()>

Broadcasts this node's joining event using the node's internal broadcast sender and metrics.

• Returns:
  Ok(()) on success, error on failure (except during shutdown).

• Usage Example:

node.broadcast_node_joining()?;

This method calls the standalone broadcast_node_joining function with the node's context.

send_block_request

pub(crate) fn send_block_request(
    &self,
    node_id: NodeIdentifier,
    included_from: BlockSeqNo,
    excluded_to: BlockSeqNo,
    at_least_n_blocks: Option<usize>,
) -> anyhow::Result<()>

Requests a range of blocks from a specified node. Reports metrics for the number of blocks requested.

• Parameters:

  • node_id: Target node to send the request.

  • included_from: Starting block sequence number (inclusive).

  • excluded_to: Ending block sequence number (exclusive).

  • at_least_n_blocks: Optional minimum number of blocks expected.

• Returns:
  Ok(()) if the request is sent successfully or an error otherwise (except during shutdown).

• Usage Example:

node.send_block_request(node_id, 50, 100, Some(10))?;

broadcast_sync_finalized

pub(crate) fn broadcast_sync_finalized(
    &self,
    block_identifier: BlockIdentifier,
    block_seq_no: BlockSeqNo,
    shared_res_address: HashMap<ThreadIdentifier, BlockIdentifier>,
) -> anyhow::Result<()>

Broadcasts a signed SyncFinalized message indicating finalized state synchronization of a block.

• Parameters:

  • block_identifier: Identifier of the finalized block.

  • block_seq_no: Sequence number of the finalized block.

  • shared_res_address: Map of thread IDs to block identifiers representing shared resources.

• Returns:
  Returns Ok(()) on successful broadcast or an error otherwise (excluding during shutdown).

• Implementation Details:

  • Converts the shared resource address map to a BTreeMap for deterministic ordering.

  • Builds a SyncFinalizedData object encapsulating the block details.

  • Attempts to retrieve block state and associated BK set for signing.

  • Signs the SyncFinalizedData envelope using BLS cryptography.

  • Broadcasts the signed envelope as a NetworkMessage::SyncFinalized.

• Usage Example:

node.broadcast_sync_finalized(block_id, seq_no, shared_res_map)?;

send_sync_from

pub(crate) fn send_sync_from(
    &self,
    node_id: NodeIdentifier,
    from_seq_no: BlockSeqNo,
) -> anyhow::Result<()>

Sends a direct SyncFrom message to a node, indicating a request to synchronize starting from a specific block sequence number.

• Parameters:

  • node_id: Target node to send the sync request.

  • from_seq_no: Starting block sequence number for synchronization.

• Returns:
  Ok(()) if the message is sent successfully, error otherwise (except during shutdown).

• Usage Example:

node.send_sync_from(peer_node_id, 123)?;

broadcast_ack

pub(crate) fn broadcast_ack(
    &self,
    ack: <Self as NodeAssociatedTypes>::Ack,
) -> anyhow::Result<()>

Broadcasts an acknowledgment (Ack) message to all nodes in the network.

• Parameters:

  • ack: The acknowledgment data to broadcast.

• Returns:
  Ok(()) on successful broadcast or an error (except during shutdown).

• Usage Example:

node.broadcast_ack(ack_message)?;

• Notes:
  This method is currently marked with #[allow(dead_code)] indicating it might not be used presently.

Important Implementation Details

• Error Handling:
  Most network operations return errors wrapped in anyhow::Result. However, when the global SHUTDOWN_FLAG is set, errors during sending or broadcasting are suppressed to allow graceful shutdown.

• BLS Signatures:
  The function broadcast_sync_finalized uses BLS cryptography to sign data envelopes before broadcasting. It retrieves block state and cryptographic keys from guarded structures to ensure thread safety.

• Metrics Reporting:
  Certain broadcasts and requests report metrics for monitoring block production and network events via BlockProductionMetrics.

• Network Message Types:
  The file operates primarily with messages of type NetworkMessage, such as BlockRequest, NodeJoining, SyncFinalized, SyncFrom, and Ack. These are part of the node's network communication protocol.

Interactions With Other System Components

• Network Communication:
  Utilizes NetDirectSender for point-to-point messaging and NetBroadcastSender for disseminating messages to all nodes.

• Node Configuration and State:
  Accesses node identifiers, thread identifiers, and configuration through the Node struct.

• Block State Repository:
  Reads block states from block_state_repository for signature operations.

• Cryptographic Primitives:
  Uses BLS signature schemes (GoshBLS) and envelopes (Envelope) for secure message signing.

• Metrics and Monitoring:
  Interfaces with BlockProductionMetrics to report network and block-related statistics.

• Shutdown Coordination:
  Checks SHUTDOWN_FLAG to avoid sending messages during system shutdown.

Diagram: Structure and Workflow of send.rs

flowchart TD
A[send_blocks_range_request]
B["broadcast_node_joining (fn)"]
C[Node Struct]
C1["broadcast_node_joining (method)"]
C2[send_block_request]
C3[broadcast_sync_finalized]
C4[send_sync_from]
C5[broadcast_ack]
A -->|uses| NetworkDirectSender
B -->|uses| NetworkBroadcastSender
C1 --> B
C2 --> A
C3 -->|uses| BlockStateRepository
C3 -->|uses| BLS Keys
C3 -->|uses| NetworkBroadcastSender
C4 -->|uses| NetworkDirectSender
C5 -->|uses| NetworkBroadcastSender

This flowchart illustrates the relationship between standalone functions and the corresponding methods in the Node struct, showing their dependencies on network senders and other components. The Node methods wrap the standalone functions and add additional logic such as metrics reporting and cryptographic signing.