Blockbook Indexer Services

Purpose

Blockbook Indexer Services address the challenge of efficiently indexing and querying blockchain data for UTXO-based blockchains within the broader Multi-Blockchain Coinstacks architecture. These services provide a robust caching layer and optimized data retrieval mechanisms to support fast transaction history queries, balance lookups, and mempool tracking. They abstract the complexities of raw blockchain data access, enabling API servers to deliver consistent and performant blockchain data to clients.

Unlike daemon nodes that maintain blockchain consensus, Blockbook indexers focus exclusively on indexing blockchain data into a queryable format, supporting REST and WebSocket APIs that serve the API servers. This separation of concerns allows the system to scale indexing workloads independently and offload heavy query operations from blockchain nodes.

Functionality

The Blockbook indexer operates as a specialized service that:

• Indexes Blockchain Data: Processes new blocks and transactions from the blockchain network, parsing and storing relevant information such as addresses, balances, and transaction metadata.

• Caches Data Efficiently: Maintains in-memory and persistent caches to accelerate frequent queries for addresses, transactions, and UTXOs.

• Manages Mempool Transactions: Tracks unconfirmed transactions in the mempool, updating clients on pending state changes.

• Exposes Query Interfaces: Provides RESTful and WebSocket endpoints that serve blockchain data to API servers and clients.

• Supports Fiat Rate Integration: Regularly queries external fiat rate providers (e.g., CoinGecko) to enrich blockchain data with up-to-date currency values.

• Handles Message Queue Binding: Uses configured message queues to synchronize data and distribute indexing workloads if needed.

• Configurable Worker Threads: Utilizes configurable numbers of mempool workers and sub-workers to parallelize mempool transaction processing.

Each blockchain coinstack includes a tailored Blockbook indexer configuration (e.g., `node/coinstacks/litecoin/indexer/config.json`) specifying blockchain-specific parameters such as RPC connection details, address formats, extended public key magic bytes, and caching policies.

Example Configuration Snippet

{
  "rpc_url": "http://localhost:8332",
  "rpc_user": "user",
  "rpc_pass": "password",
  "mempool_workers": 8,
  "mempool_sub_workers": 2,
  "block_addresses_to_keep": 300,
  "fiat_rates": "coingecko",
  "fiat_rates_params": "{\"url\": \"https://api.coingecko.com/api/v3\", \"coin\": \"litecoin\", \"periodSeconds\": 900}"
}

This configuration directs the indexer to connect to the blockchain RPC node, manage mempool processing with specified concurrency, and enrich data with fiat currency rates updated every 900 seconds.

Integration with Parent Topic and Other Subtopics

Blockbook Indexer Services integrate tightly with other components in the Multi-Blockchain Coinstacks ecosystem:

• Daemon Nodes: The indexer subscribes to blockchain data published by daemon nodes, typically via RPC or message queues. It relies on daemon nodes for the authoritative blockchain state but handles all indexing logic independently. This separation allows daemon nodes to focus on consensus and network participation.

• API Servers: API servers consume data from the Blockbook indexer to serve client requests for transaction history, balances, and real-time events. The API layer queries the indexer’s REST and WebSocket interfaces rather than directly accessing blockchain nodes, improving response times and reducing load on daemon nodes.

• WebSocket Event Subscription: The indexer's WebSocket output feeds into the subscription management system, enabling real-time notifications to clients about new transactions and blocks relevant to their subscribed addresses.

• Deployment Automation: The indexer is deployed as a Kubernetes StatefulSet with health and readiness probes configured via shell scripts to ensure it remains synchronized and responsive. Pulumi scripts automate deployment and configuration, using the indexer’s config files to instantiate the service correctly per blockchain.

The Blockbook indexer thus forms a critical middleware layer bridging raw blockchain data and the unified API surface, providing enhanced performance, scalability, and modularity.

Diagram

flowchart TD
    subgraph Blockchain Network
        Blockchain[Blockchain Nodes]
    end

    Daemon[Daemon Node]
    Indexer[Blockbook Indexer Service]
    APIServer[API Server]
    Client[Client Application]

    Blockchain -->|RPC / Message Queue| Daemon
    Daemon -->|Block & Tx Data| Indexer
    Indexer -->|REST / WebSocket API| APIServer
    APIServer -->|REST API Calls| Client
    APIServer -->|WebSocket Events| Client
    Indexer -->|Real-time Events| WebSocketMgr[WebSocket Event Subscription]

    style Blockchain fill:#f9f,stroke:#333,stroke-width:1px
    style Daemon fill:#bbf,stroke:#333,stroke-width:1px
    style Indexer fill:#bfb,stroke:#333,stroke-width:1px
    style APIServer fill:#ffb,stroke:#333,stroke-width:1px
    style Client fill:#fbf,stroke:#333,stroke-width:1px
    style WebSocketMgr fill:#fbb,stroke:#333,stroke-width:1px

This flowchart illustrates the data pipeline from the blockchain network through daemon nodes to the Blockbook indexer service, which serves the API servers. Real-time events from the indexer are propagated to clients via the WebSocket subscription manager, enabling live updates.

By focusing on efficient blockchain data indexing and caching, Blockbook Indexer Services play a pivotal role in delivering fast, reliable, and scalable blockchain data access within the multi-blockchain architecture. Their modular design and configurable parameters allow seamless adaptation to multiple UTXO-based blockchains, complementing daemon nodes and API servers to form a cohesive system.