serialize_block.rs

Overview

This file provides functionality for reflecting and archiving blocks and their related data (accounts, transactions, messages) into a database system. It primarily processes a tvm_block extracted from a cryptographic envelope, serializes its contents, prepares archival structures, and stores them in a document-oriented database. The file handles intricate interactions between blockchain block data, accounts, transactions, and messages, ensuring consistent representation and traceability of blockchain state changes.

Key responsibilities include:

• Extracting and serializing blocks, transactions, accounts, and messages.

• Preparing archival database records for these blockchain entities.

• Managing transaction and message ordering within blocks.

• Handling deleted accounts and maintaining consistency with shard state.

• Integrating cryptographic proofs where applicable.

• Logging detailed timing and process information.

Detailed Descriptions

Constants and Static References

• ACCOUNT_NONE_HASH: A static hash representing the serialization of a default (non-existent) account. Used to detect deleted accounts.

• MINTER_ADDRESS: A static standard address representing the minter with a zeroed address body.

Functions

reflect_block_in_db

pub fn reflect_block_in_db(
    archive: Arc<Mutex<dyn DocumentsDb>>,
    envelope: Envelope<GoshBLS, AckiNackiBlock>,
    raw_block: Option<Vec<u8>>,
    shard_state: Arc<ShardStateUnsplit>,
    transaction_traces: &mut HashMap<UInt256, Vec<EngineTraceInfoData>, RandomState>,
) -> anyhow::Result<()>

Purpose:
Processes a block encapsulated in an envelope, reflects its data into the database by preparing and storing archived versions of the block, transactions, accounts, and messages.

Parameters:

• archive: Thread-safe reference to the document database interface for storing blockchain data.

• envelope: Cryptographic envelope wrapping the block data.

• raw_block: Optional raw byte representation of the block.

• shard_state: Current state snapshot of the shard, used to access accounts.

• transaction_traces: Mutable map of transaction hashes to engine trace data for diagnostics.

Returns:

• anyhow::Result<()>: Returns Ok(()) on success or an error if any processing step fails.

Usage Example:

reflect_block_in_db(
    archive.clone(),
    envelope,
    Some(raw_block_bytes),
    shard_state.clone(),
    &mut transaction_traces,
)?;

Implementation Details:

• Serializes the block into a BOC (Bag of Cells) format and calculates block and file hashes.

• Reads block extra data to extract account blocks and their transactions.

• Differentiates between changed and deleted accounts by comparing state hashes.

• Prepares and orders transactions and messages for archival.

• Updates account archival records, handling both changed and deleted accounts.

• Stores transactions, accounts, messages, and the block itself into the database.

• Logs timing information for performance monitoring.

prepare_messages_from_transaction

pub(crate) fn prepare_messages_from_transaction(
    transaction: &Transaction,
    block_id: UInt256,
    transaction_id: UInt256,
    transaction_index: &str,
    block_root_for_proof: Option<&Cell>,
    messages: &mut HashMap<UInt256, ArchMessage>,
) -> anyhow::Result<()>

Purpose:
Extracts and prepares message archive structures from a given transaction, including both inbound and outbound messages.

Parameters:

• transaction: Reference to the transaction to process.

• block_id: Unique identifier hash of the block containing the transaction.

• transaction_id: Unique identifier hash of the transaction.

• transaction_index: String representing the transaction's position/order in the block.

• block_root_for_proof: Optional reference to the block root cell for generating cryptographic proofs.

• messages: Mutable map collecting prepared message archive structures keyed by message hash.

Returns:

• anyhow::Result<()>: Success or error in message preparation.

Implementation Details:

• Processes the inbound message if present and not restricted by configuration.

• Iterates over outbound messages, preparing archive structures with chain ordering.

• Uses helper function prepare_message_archive_struct to serialize messages.

• Indexes messages based on creation logical time to maintain ordering.

prepare_message_archive_struct

pub(crate) fn prepare_message_archive_struct(
    message_cell: Cell,
    message: Message,
    block_root_for_proof: Option<&Cell>,
    block_id: UInt256,
    transaction_id: Option<UInt256>,
    transaction_now: Option<u32>,
) -> tvm_types::Result<ArchMessage>

Purpose:
Creates an archival database record for a message, optionally including cryptographic proof data.

Parameters:

• message_cell: BOC cell containing the raw message data.

• message: Deserialized message object.

• block_root_for_proof: Optional block root cell for proof generation.

• block_id: Identifier of the block containing the message.

• transaction_id: Optional transaction ID associated with the message.

• transaction_now: Optional timestamp associated with the transaction.

Returns:

• tvm_types::Result<ArchMessage>: Archival message structure for database insertion.

Usage:
Called internally during transaction processing to generate message records.

prepare_transaction_archive_struct

pub(crate) fn prepare_transaction_archive_struct(
    tr_cell: Cell,
    transaction: Transaction,
    block_root: &Cell,
    block_id: UInt256,
    workchain_id: i32,
    add_proof: bool,
    _trace: Option<Vec<EngineTraceInfoData>>,
) -> anyhow::Result<ArchTransaction>

Purpose:
Prepares an archival structure for a transaction including optional cryptographic proof.

Parameters:

• tr_cell: Cell containing the transaction data.

• transaction: The transaction object.

• block_root: Reference to the block root cell for proof generation.

• block_id: Block hash identifier.

• workchain_id: Workchain identifier where the transaction occurred.

• add_proof: Flag to indicate if proof should be added.

• _trace: Optional engine trace data (currently unused).

Returns:

• anyhow::Result<ArchTransaction>: Prepared archival transaction structure.

Implementation Details:

• Serializes transaction to BOC format.

• Optionally generates proof using the block root.

• Converts serialization set into archival record.

prepare_account_archive_struct

pub(crate) fn prepare_account_archive_struct(
    account: Account,
    prev_account_state: Option<Account>,
    last_trans_chain_order: Option<String>,
    last_trans_hash: UInt256,
    dapp_id: Option<UInt256>,
) -> anyhow::Result<ArchAccount>

Purpose:
Constructs an archive record representing an account's current state.

Parameters:

• account: Current account state object.

• prev_account_state: Optional previous account state for comparison.

• last_trans_chain_order: Optional string indicating last transaction's chain order.

• last_trans_hash: Hash of the last transaction associated with this account.

• dapp_id: Optional decentralized application identifier linked to the account.

Returns:

• anyhow::Result<ArchAccount>: Prepared archival account structure.

Usage:
Used to archive changed accounts reflecting the latest blockchain state.

prepare_deleted_account_archive_struct

pub(crate) fn prepare_deleted_account_archive_struct(
    account_id: AccountId,
    workchain_id: i32,
    prev_account_state: Option<Account>,
    last_trans_chain_order: Option<String>,
) -> anyhow::Result<ArchAccount>

Purpose:
Prepares an archival record for an account that has been deleted (i.e., removed from the state).

Parameters:

• account_id: Identifier of the deleted account.

• workchain_id: Workchain identifier where the account existed.

• prev_account_state: Optional previous state of the account before deletion.

• last_trans_chain_order: Optional last transaction chain order string.

Returns:

• anyhow::Result<ArchAccount>: Archival record marking the account as deleted.

prepare_block_archive_struct

pub(crate) fn prepare_block_archive_struct(
    envelope: Envelope<GoshBLS, AckiNackiBlock>,
    block_root: &Cell,
    boc: &[u8],
    file_hash: &UInt256,
    block_order: String,
) -> anyhow::Result<ArchBlock>

Purpose:
Generates an archival record for a block including metadata, cryptographic signatures, message lists, and serialized block data.

Parameters:

• envelope: The cryptographic envelope containing the block.

• block_root: Root cell of the block.

• boc: Raw bytes of the block BOC.

• file_hash: Hash of the block file bytes.

• block_order: String representing the block's ordering key.

Returns:

• anyhow::Result<ArchBlock>: Archival block record ready for storage.

Implementation Details:

• Extracts producer and thread ID metadata.

• Serializes block data and signatures.

• Collects lists of inbound and outbound message hashes.

• Calculates total transaction count in the block.

• Sets chain order and parent block reference.

block_index

pub(crate) fn block_index(block: &Block) -> anyhow::Result<String>

Purpose:
Computes a unique string index for a block based on its generation time, shard prefix, and sequence number.

Parameters:

• block: Reference to the block structure.

Returns:

• anyhow::Result<String>: Computed block index string.

Usage:
Used to order transactions and blocks consistently within the archival system.

u64_to_string

pub fn u64_to_string(value: u64) -> String

Purpose:
Encodes a 64-bit unsigned integer into a hexadecimal string with a prefixed length indicator.

Parameters:

• value: The 64-bit unsigned integer to encode.

Returns:

• String: Encoded string representation.

Important Implementation Details and Algorithms

• Account Change Detection:
  Accounts are identified as changed or deleted by comparing their old and new state hashes against a predefined ACCOUNT_NONE_HASH. This hash is derived from a default empty account serialization.

• Transaction Ordering:
  Transactions are ordered using a tuple of (logical_time, account_address) to maintain a deterministic sequence within the block.

• Message Chain Ordering:
  Messages are ordered within transactions using a combination of the transaction index and message-specific indexes to track inbound and outbound message relationships.

• Proof Generation:
  Optional cryptographic proofs are generated for transactions and messages using the block root cell, enhancing data integrity verification.

• Database Interaction:
  Data is batched and committed to the document database through thread-safe Mutex locking on the archive interface. Transactions, accounts, messages, and blocks are stored via distinct put_* methods.

• Performance Logging:
  The function logs elapsed times for major processing steps to facilitate performance monitoring and debugging.

Interactions with Other System Components

• Database Layer (DocumentsDb):
  This file depends heavily on the database interface to store serialized blockchain entities. It uses the put_transactions, put_accounts, put_messages, and put_block methods to persist data.

• Blockchain Data Types (tvm_block and tvm_types):
  Uses core data types for blocks, accounts, transactions, and messages for de/serialization and state reading.

• Cryptographic Envelope (Envelope<GoshBLS, AckiNackiBlock>):
  The block data is wrapped in a cryptographic envelope that provides signatures and aggregated signatures for verification and archival.

• Shard State (ShardStateUnsplit):
  Accesses the current shard state to read account data and verify account changes.

• BLS Signature System (GoshBLS):
  Integrates with BLS signature logic for block signature handling.

• Trace Data (EngineTraceInfoData):
  Optionally uses trace data from the block producer's engine for enhanced diagnostics, though currently this is commented out.

Visual Diagram: File Structure and Function Relationships

flowchart TD
A[reflect_block_in_db] --> B[prepare_messages_from_transaction]
A --> C[prepare_transaction_archive_struct]
A --> D[prepare_account_archive_struct]
A --> E[prepare_deleted_account_archive_struct]
A --> F[prepare_block_archive_struct]
B --> G[prepare_message_archive_struct]
A -->|calls| H[block_index]
H --> I[u64_to_string]
style A fill:#f9f,stroke:#333,stroke-width:2px
style B fill:#bbf,stroke:#333,stroke-width:1.5px
style C fill:#bbf,stroke:#333,stroke-width:1.5px
style D fill:#bbf,stroke:#333,stroke-width:1.5px
style E fill:#bbf,stroke:#333,stroke-width:1.5px
style F fill:#bbf,stroke:#333,stroke-width:1.5px
style G fill:#afa,stroke:#333,stroke-width:1.5px
style H fill:#fc9,stroke:#333,stroke-width:1.5px
style I fill:#fc9,stroke:#333,stroke-width:1.5px

Notes on Usage and Integration

• The file is designed to be invoked when a new block is ready to be reflected into the archival database.

• The reflect_block_in_db function acts as the central entry point, orchestrating calls to helper functions that prepare archival data.

• The file depends on configuration flags such as store_events_only to conditionally process message data.

• The archival data structures (ArchBlock, ArchTransaction, ArchAccount, ArchMessage) comply with the database schema and serialization requirements.

• Timing and tracing features are integrated via tracing macros to support observability during block reflection.

• The file uses concurrency-safe wrappers (Arc, Mutex) to interact with shared database resources safely.

This file is critical for maintaining a reliable, queryable, and traceable record of blockchain state transitions in the archival database. It bridges raw blockchain data structures and the persistent storage layer while preserving cryptographic integrity and ordering semantics.