readiness.sh

Overview

`readiness.sh` is a shell script designed as a **readiness probe** for an Ethereum Virtual Machine (EVM) compatible blockchain node daemon—specifically targeting Arbitrum in this instance. Its primary purpose is to verify whether the blockchain node is sufficiently synchronized and ready to accept traffic or API requests.

This script performs a series of checks including:

Detecting if readiness probing is temporarily disabled via a sentinel file.
Querying the local node's sync status via JSON-RPC (eth_syncing).
If the node reports "not syncing," fetching the current block height.
Comparing the local block height against multiple external reference nodes to validate sync accuracy.
Enforcing a block height tolerance threshold to decide readiness.
Exiting with appropriate status codes understood by Kubernetes for container orchestration.

The script ensures Kubernetes only routes traffic to nodes that are properly synchronized, improving reliability and user experience.

Detailed Explanation

Constants and Variables

Name	Purpose
`DISABLE_READINESS_PROBE`	Path to a file (`/data/disable_readiness`) which, if present, disables readiness checks.
`BLOCK_HEIGHT_TOLERANCE`	Integer tolerance for block height difference (set to 15 blocks) allowed between node and network.

Step-by-step Functionality

1. Disable Readiness Probe Check

if [[ -f "$DISABLE_READINESS_PROBE" ]]; then
  echo "readiness probe disabled"
  exit 0
fi

Purpose: Allows operators to manually disable readiness checks by creating the file /data/disable_readiness.
Effect: If the file exists, the script exits with status 0, signaling readiness regardless of node status.

2. Source External Utility Script

source /evm.sh

Purpose: Loads shared functions and variables related to EVM chains.
Key Function Used: get_best_reference_block_number — queries multiple external RPC endpoints to find the highest block number reported by trusted reference nodes.
Note: The actual implementation of /evm.sh is external and assumed to provide critical utility functions.

3. Query Local Node Sync Status

ETH_SYNCING=$(curl -sf -d '{"jsonrpc":"2.0","method":"eth_syncing","params":[],"id":1}' \
  -H 'Content-Type: application/json' http://localhost:8547) || exit 1

RPC Method: eth_syncing returns the sync status of the node.
Behavior:
- If the node is syncing, .result will be an object containing sync info.
- If fully synced, .result is false.
On failure: Curl errors or no response cause the script to exit with 1 (not ready).

4. Parse Sync Status

SYNCING=$(echo $ETH_SYNCING | jq -r '.result')

Extracts the value of .result from the JSON response.
Expected values:
- false (string) → node reports fully synced
- Any JSON object (syncing data) → node is syncing

5. If Not Syncing, Validate Block Height Against Reference Nodes

if [[ $SYNCING == false ]]; then
  eth_blockNumber=$(curl -sf -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' \
    -H 'Content-Type: application/json' http://localhost:8547) || exit 1

  current_block_number_hex=$(echo $eth_blockNumber | jq -r '.result')
  current_block_number=$(($current_block_number_hex))

  best_reference_block_number=$(get_best_reference_block_number https://arb1.arbitrum.io/rpc https://arbitrum-one.publicnode.com https://arb-mainnet.g.alchemy.com/v2/demo)

  reference_validation daemon $current_block_number $best_reference_block_number $BLOCK_HEIGHT_TOLERANCE
fi

Fetch Local Block Number:
Calls eth_blockNumber RPC method to get the current block number in hex, converts to decimal.
Get Best Reference Block Number:
Calls get_best_reference_block_number (from /evm.sh) passing multiple external Arbitrum RPC endpoints. This function returns the highest block number among these external reference nodes.
Validate:
Calls reference_validation function (presumably also from /evm.sh) with parameters:
- daemon: the caller type
- current_block_number: local node block height
- best_reference_block_number: highest external block height
- BLOCK_HEIGHT_TOLERANCE: acceptable block height difference
Purpose: Ensures local node is not falsely reporting synced when it lags significantly behind the network.

6. If Syncing or Validation Fails

echo "daemon is still syncing"
exit 1

If the node reports syncing or the validation fails (implicit by not exiting earlier), the script prints a message and exits with 1.
This exit code signals to Kubernetes that the pod is not ready.

Usage Example

This script is typically run by Kubernetes readiness probe configured in the pod spec for the blockchain daemon container, e.g.:

readinessProbe:
  exec:
    command:
    - /bin/bash
    - /path/to/readiness.sh
  initialDelaySeconds: 10
  periodSeconds: 15
  failureThreshold: 3

Operators can manually disable readiness probing by creating `/data/disable_readiness` inside the container or volume mount.

Important Implementation Details

Robustness: The script uses curl -sf to fail silently on network errors and exits with non-zero to indicate failure.
JSON Parsing: Uses jq to extract JSON fields reliably.
Block Height Conversion: Converts hex block numbers to decimal for numeric comparison.
Cross-Validation: Uses multiple external RPC nodes to prevent false readiness reporting.
Tolerance Setting: The BLOCK_HEIGHT_TOLERANCE of 15 blocks accounts for network propagation delays and minor lag.

Interaction with Other System Components

Kubernetes: The script is executed periodically as a readiness probe to inform Kubernetes whether the node container is ready to accept traffic.
Local Node: Communicates with the local blockchain node RPC interface on localhost:8547.
Reference Nodes: Queries multiple external Arbitrum public RPC endpoints to verify sync status.
Utility Script /evm.sh: Provides shared functions like get_best_reference_block_number and reference_validation.
Disable File: The presence of /data/disable_readiness allows manual override of readiness checks.

Mermaid Diagram - Script Flowchart

flowchart TD
  Start[Start] --> CheckDisableFile{Is /data/disable_readiness present?}
  CheckDisableFile -- Yes --> Disabled[Print "readiness probe disabled"\nExit 0]
  CheckDisableFile -- No --> QuerySync[Query eth_syncing RPC]
  QuerySync --> ParseSync[Parse .result from eth_syncing]
  ParseSync --> IsSyncing{Is syncing == false?}
  IsSyncing -- No --> NotReady[Print "daemon is still syncing"\nExit 1]
  IsSyncing -- Yes --> QueryBlockNumber[Query eth_blockNumber RPC]
  QueryBlockNumber --> ConvertHex[Convert hex block number to decimal]
  ConvertHex --> GetReference[Call get_best_reference_block_number with external RPCs]
  GetReference --> Validate[Call reference_validation with local & reference blocks]
  Validate --> ExitSuccess[Exit 0 if validation passes]
  Validate --> ExitFail[Exit 1 if validation fails]

Summary

The `readiness.sh` script is a critical Kubernetes readiness probe tailored for EVM-compatible blockchain daemons such as Arbitrum. It performs a multi-step synchronization check by querying the local node and validating against trusted external reference nodes, enforcing a block height tolerance to ensure the node is truly ready to serve. The script’s design emphasizes fail-fast behavior, modularity via sourced utility functions, and operational flexibility through the disable file mechanism. This approach helps maintain high availability and correct traffic routing in blockchain node deployments.