liveness.sh

Overview

`liveness.sh` is a Bash script designed to serve as a **liveness probe** for a daemon interacting with an Ethereum-like blockchain node. Its primary purpose is to verify whether the daemon is actively progressing by checking the blockchain's latest block number and ensuring it increments over time. This helps container orchestration platforms (e.g., Kubernetes) or monitoring systems detect if the daemon is healthy or stalled.

The script performs the following key functions:

• Optionally disables the liveness check if a specific file exists.

• Queries the local Ethereum JSON-RPC endpoint to fetch the current latest block number.

• Compares the current block number with a previously recorded number stored on disk.

• Determines if the daemon is "running" (block number increases) or "stalled" (block number not increasing).

• Exits with appropriate codes (0 for healthy, 1 for unhealthy) for integration in liveness probes.

Detailed Explanation

Constants and Files

Script Workflow

#!/bin/bash

DISABLE_LIVENESS_PROBE=/data/disable_liveness

if [[ -f "$DISABLE_LIVENESS_PROBE" ]]; then
  echo "liveness probe disabled"
  exit 0
fi

FILE=/data/.block_number

ETH_BLOCK_NUMBER=$(curl -sf -d '{"jsonrpc":"2.0","method":"eth_blockNumber","params":[],"id":1}' -H 'Content-Type: application/json' http://localhost:8547) || exit 1

CURRENT_BLOCK_NUMBER_HEX=$(echo $ETH_BLOCK_NUMBER | jq -r '.result')
CURRENT_BLOCK_NUMBER=$(($CURRENT_BLOCK_NUMBER_HEX))

if [[ ! -f "$FILE" ]]; then
  echo $CURRENT_BLOCK_NUMBER > $FILE
  exit 1
fi

PREVIOUS_BLOCK_NUMBER=$(cat $FILE)
echo $CURRENT_BLOCK_NUMBER > $FILE

if (( $CURRENT_BLOCK_NUMBER > $PREVIOUS_BLOCK_NUMBER )); then
  echo "daemon is running"
  exit 0
fi

echo "daemon is stalled"
exit 1

Step-by-step Breakdown

1. Disable Check
   Checks if the file /data/disable_liveness exists. If yes, the script outputs "liveness probe disabled" and exits with status 0 (success). This allows manual or external disabling of the liveness probe.

2. Fetch Current Block Number
   Sends a JSON-RPC request to local Ethereum node at http://localhost:8547 to get the latest block number (eth_blockNumber method).

   • Uses curl -sf for silent fail — if the request fails, the script immediately exits with status 1 (failure).

   • Parses the JSON response using jq to extract the .result field, which is a hex string representing the block number.

   • Converts the hex string to a decimal number for comparison.

3. Initialize Block Number File
   If the file /data/.block_number does not exist, the script writes the current block number into it and exits with status 1. This initial exit signals the liveness check should re-run later to confirm progress.

4. Compare Block Numbers
   Reads the previous block number from /data/.block_number.
   Updates the file with the current block number.
   Checks if the current block number is greater than the previous:

   • If yes, prints "daemon is running" and exits with 0 (healthy).

   • Otherwise, prints "daemon is stalled" and exits with 1 (unhealthy).

Usage Example

Run the script manually to perform a liveness check:

bash liveness.sh

Expected outputs:

• If disabled:

  liveness probe disabled
  

• On first run (no previous block number stored):

  (no output, exit code 1)

• If daemon is progressing:

  daemon is running
  

• If daemon is stalled:

  daemon is stalled
  

Important Implementation Details

• JSON-RPC Request: The script uses a direct curl POST request with JSON body to query the Ethereum node's block number. This is a lightweight and standard way to interface with Ethereum-compatible nodes.

• Hexadecimal Handling: Ethereum block numbers are returned as hexadecimal strings (e.g., "0x10d4f"). The script converts these to decimal integers using shell arithmetic for numerical comparison.

• State Persistence: The script persists the last seen block number in the file /data/.block_number to maintain state across invocations. This simple file-based persistence allows liveness checks without needing a database or in-memory state.

• Exit Codes: Exit code 0 indicates a healthy daemon state, exit code 1 signals an unhealthy or indeterminate state.

Integration and Interaction with System

• This script is intended to be used as a Kubernetes liveness probe or similar health check mechanism for a daemon that processes blockchain data.

• It depends on the local Ethereum node running and exposing its JSON-RPC interface at http://localhost:8547.

• The script expects a writable volume mounted at /data for storing the .block_number file and optionally the disable_liveness flag file.

• Container orchestrators can use the script's exit status to restart or alert on daemon health.

Visual Diagram: Workflow Flowchart

flowchart TD
    A[Start liveness.sh] --> B{Is disable_liveness file present?}
    B -- Yes --> C[Print "liveness probe disabled" and exit 0]
    B -- No --> D[Send JSON-RPC request to get block number]
    D --> E{Request successful?}
    E -- No --> F[Exit 1 (failure)]
    E -- Yes --> G[Parse block number hex to decimal]
    G --> H{Does /data/.block_number exist?}
    H -- No --> I[Write current block number to file and exit 1]
    H -- Yes --> J[Read previous block number from file]
    J --> K[Update file with current block number]
    K --> L{Is current block number > previous?}
    L -- Yes --> M[Print "daemon is running" and exit 0]
    L -- No --> N[Print "daemon is stalled" and exit 1]

Summary

The `liveness.sh` script is a simple but effective liveness probe for monitoring an Ethereum daemon’s operational health by verifying blockchain synchronization progress. Its straightforward approach with JSON-RPC queries, file-based state persistence, and clear exit codes makes it ideal for integration in containerized environments requiring automated health checks.