readiness.sh

Overview

`readiness.sh` is a shell script implementing a **readiness probe** for an Ethereum-compatible blockchain node daemon (EVM chain). Its primary purpose is to determine if the local blockchain node is sufficiently synchronized and network-connected to be considered "ready" to serve requests. This script is typically executed by Kubernetes as part of container health checks to control traffic routing and load balancing.

The script performs the following key functions:

• Checks for a disable file that can temporarily bypass the readiness check.

• Queries the local Ethereum JSON-RPC endpoint to determine syncing status and peer connectivity.

• If synced and connected to peers, obtains the current block height.

• Cross-validates the local node's block height against multiple external reference RPC nodes to confirm synchronization status within a tolerance window.

• Exits with status 0 if ready, otherwise 1 to signal not ready.

This ensures that only fully synchronized and network-connected nodes receive traffic, improving reliability and consistency of blockchain services.

Detailed Explanation

Variables and Constants

External Dependencies

• curl: For making HTTP POST requests to JSON-RPC endpoints.

• jq: For parsing JSON responses.

• /evm.sh: An external shell script sourced to provide utility functions:

  • get_best_reference_block_number: Queries multiple reference RPC endpoints and returns the highest block number.

  • reference_validation: Compares local block number against reference block number with tolerance.

Script Execution Flow

1. Disable File Check

   if [[ -f "$DISABLE_READINESS_PROBE" ]]; then
     echo "readiness probe disabled"
     exit 0
   fi
   

   If the file /data/disable_readiness exists, the script immediately exits with 0, signaling readiness disabled.

2. Source Utility Script

   source /evm.sh
   

   This loads shared functions for EVM chain validation.

3. Constants

   BLOCK_HEIGHT_TOLERANCE=15
   

   This sets the allowed lag in blocks.

4. Query Syncing Status

   ETH_SYNCING=$(curl -sf -d '{"jsonrpc":"2.0","method":"eth_syncing","params":[],"id":1}' -H 'Content-Type: application/json' http://localhost:8545) || exit 1
   SYNCING=$(echo $ETH_SYNCING | jq -r '.result')
   

   Calls eth_syncing method. If node is syncing, .result will contain syncing details; if fully synced, it returns false.

5. Query Peer Count

   NET_PEER_COUNT=$(curl -sf -d '{"jsonrpc":"2.0","method":"net_peerCount","params":[],"id":1}' -H 'Content-Type: application/json' http://localhost:8545) || exit 1
   PEER_COUNT_HEX=$(echo $NET_PEER_COUNT | jq -r '.result')
   PEER_COUNT=$(($PEER_COUNT_HEX))
   

   Retrieves number of connected peers as a hex string and converts to integer.

6. Evaluate Sync and Peer Status

   • If SYNCING is false and there is at least one peer, proceed to check block height.

   • If SYNCING is false but no peers, exit 1 indicating not ready.

   • If still syncing, exit 1.

7. Obtain Current Block Number

   eth_blockNumber=$(curl -sf -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' -H 'Content-Type: application/json' http://localhost:8545) || exit 1
   current_block_number_hex=$(echo $eth_blockNumber | jq -r '.result')
   current_block_number=$(($current_block_number_hex))
   

8. Get Best Reference Block Number

   best_reference_block_number=$(get_best_reference_block_number https://polygon-rpc.com https://polygon-bor.publicnode.com https://polygon-mainnet.g.alchemy.com/v2/demo)
   

   Queries multiple external RPC endpoints and returns the highest block number observed.

9. Validate Local Block Number

   reference_validation daemon $current_block_number $best_reference_block_number $BLOCK_HEIGHT_TOLERANCE
   

   Ensures local block is within tolerance of the highest reference. Exits non-zero if validation fails.

10. Print Status and Exit

    • On success: echo "daemon is synced, with $PEER_COUNT peers" and exit 0.

    • If no peers: echo "daemon is synced, but has no peers" and exit 1.

    • If syncing: echo "daemon is still syncing" and exit 1.

Important Implementation Details

• Tolerance-based Sync Validation: Instead of requiring exact block height match, it allows configurable lag (BLOCK_HEIGHT_TOLERANCE) to accommodate network delays.

• Reference Nodes Cross-validation: Helps avoid false positives if local node reports sync but is behind the network.

• Peer Connectivity Check: Ensures the node is actively participating in the network.

• Disable File Mechanism: Allows operators to bypass readiness checks for maintenance or debugging.

• Error Handling: Uses curl -sf to fail silently on network errors and exits with 1 immediately, signaling probe failure.

• Hexadecimal to Decimal Conversion: Ethereum RPC returns numeric values as hex strings; these are converted to integers for comparisons.

Interaction with Other System Components

• Kubernetes: Configured as a readiness probe in the pod spec of the blockchain node container. Kubernetes uses the script's exit code (0 or 1) to decide if the pod is ready to receive requests.

• Local Ethereum Node: Accesses the local node's JSON-RPC interface (default at http://localhost:8545) to obtain syncing, peer count, and block height.

• Reference RPC Nodes: Queries external trusted RPC endpoints (e.g., Polygon RPC URLs) to get authoritative block heights for validation.

• Utility Script (/evm.sh): Provides shared functions for querying reference nodes and validation logic.

Usage Example

To manually test the readiness probe:

chmod +x readiness.sh
./readiness.sh

• Output "readiness probe disabled" and exit 0 if the disable file exists.

• Output "daemon is synced, with N peers" and exit 0 if node is ready.

• Output "daemon is still syncing" or "daemon is synced, but has no peers" and exit 1 otherwise.

In Kubernetes, this script runs automatically at configured intervals.

Mermaid Diagram: Flowchart of readiness.sh

flowchart TD
    Start[Start Script] --> CheckDisable{Disable File Present?}
    CheckDisable -- Yes --> Disabled[Print "readiness probe disabled"\nExit 0]
    CheckDisable -- No --> SourceEVM[Source /evm.sh]
    SourceEVM --> QuerySync[Query eth_syncing]
    QuerySync --> ParseSync[Parse syncing status]
    ParseSync --> QueryPeers[Query net_peerCount]
    QueryPeers --> ParsePeers[Parse peer count]
    ParseSync --> IsSynced{SYNCING == false?}
    IsSynced -- No --> Syncing[Print "daemon is still syncing"\nExit 1]
    IsSynced -- Yes --> HasPeers{PEER_COUNT > 0?}
    HasPeers -- No --> NoPeers[Print "daemon is synced, but has no peers"\nExit 1]
    HasPeers -- Yes --> GetBlockNumber[Query eth_blockNumber]
    GetBlockNumber --> ParseBlockNumber[Parse current block number]
    ParseBlockNumber --> GetRefBlock[Call get_best_reference_block_number]
    GetRefBlock --> ValidateRef[Call reference_validation]
    ValidateRef --> Valid?{Validation passed?}
    Valid? -- No --> RefFail[Exit 1]
    Valid? -- Yes --> Ready[Print "daemon is synced, with PEER_COUNT peers"\nExit 0]

Summary

The `readiness.sh` script is a critical component ensuring that an Ethereum-compatible blockchain node is fully synchronized and network-connected before being considered ready for traffic. It integrates tightly with Kubernetes readiness probes, local node RPC endpoints, and external reference nodes to provide robust validation. The script's design emphasizes fail-fast semantics, configurable tolerance, and operational flexibility via a disable file mechanism.

This approach helps maintain high availability and consistency for blockchain services running in containerized environments.

End of documentation for readiness.sh