liveness.sh

Overview

`liveness.sh` is a Bash script designed to serve as a liveness probe for a daemon that interacts with an Ethereum node running locally (via JSON-RPC on port 8545). The script's primary function is to check whether the daemon is actively processing new Ethereum blocks by comparing the current Ethereum block number to the last recorded block number. If the block number advances, it indicates the daemon is running correctly; if not, it signals a potentially stalled daemon.

Additionally, the script supports being disabled by the presence of a specific file (`/data/disable_liveness`), allowing operators to bypass the liveness check if needed.

Detailed Explanation

Script Workflow

1. Check for Disable Flag
   The script first checks if the file /data/disable_liveness exists. If so, it outputs liveness probe disabled and exits with status code 0, effectively disabling the liveness check.

2. Fetch Current Ethereum Block Number
   The script queries the local Ethereum node via JSON-RPC to get the latest block number by sending a POST request to http://localhost:8545 with the method eth_blockNumber.

3. Parse and Convert Block Number
   The returned block number is in hexadecimal format (e.g., "0x10d4f"). The script extracts the .result field using jq and converts the hex string to a decimal number for comparison.

4. Compare with Previous Block Number

   • If this is the first run (no saved block number file /data/.block_number), the script saves the current block number and exits with code 1 (indicating not ready/stalled on startup).

   • On subsequent runs, it reads the previous block number from the file and compares it with the current block number.

   • If the current block number is greater, it means new blocks have been processed and the daemon is running; the script outputs daemon is running and exits 0.

   • If the current block number is not greater, the daemon is considered stalled; the script outputs daemon is stalled and exits 1.

Script Breakdown

Variables

• DISABLE_LIVENESS_PROBE
  Path to the file that disables the liveness probe when present (/data/disable_liveness).

• FILE
  Path to the file where the last known block number is stored (/data/.block_number).

• ETH_BLOCK_NUMBER
  Raw JSON response from the Ethereum node containing the block number.

• CURRENT_BLOCK_NUMBER_HEX
  The block number extracted from the JSON response, in hexadecimal.

• CURRENT_BLOCK_NUMBER
  The block number converted from hex to decimal.

• PREVIOUS_BLOCK_NUMBER
  The previously saved block number read from the file.

Important Commands and Tools

• curl
  Used to send the JSON-RPC request to the Ethereum node.

• jq
  Used to parse JSON and extract the .result field.

• Arithmetic expansion $((...))
  Used to convert hex to decimal and compare block numbers.

Usage Example

Assuming the script is executable and the local Ethereum node is running and accessible at `http://localhost:8545`:

./liveness.sh

Possible outputs:

• liveness probe disabled (exit 0): Liveness check disabled by presence of disable file.

• daemon is running (exit 0): Daemon is actively processing new blocks.

• daemon is stalled (exit 1): No new blocks processed; daemon may be stalled.

• No output (exit 1): Could indicate failure to fetch block number.

Implementation Details and Algorithms

• The script leverages the standard Ethereum JSON-RPC method eth_blockNumber to retrieve the latest block number.

• It uses a simple persistent state mechanism with a file (/data/.block_number) to remember the last known block number across runs.

• The logic is straightforward: if the block number advances, the daemon is live; otherwise, it is stalled.

• The presence of a disabling file provides an operational override to bypass the liveness check, useful during maintenance or debugging.

Interaction with Other System Components

• Ethereum Node (Local JSON-RPC Server):
  The script depends on a locally running Ethereum node exposing JSON-RPC on http://localhost:8545. It must be accessible for the script to function.

• Daemon Being Monitored:
  The daemon presumably processes Ethereum blocks and updates state accordingly. This script checks its health indirectly by monitoring block progression.

• File System (/data directory):
  Used to store the disable flag file and the last block number. This directory must be writable and persistent for correct operation.

• Kubernetes or Container Orchestration Systems (typical use case):
  This script is likely used as a Kubernetes liveness probe script to determine pod health and restart if stalled.

Visual Diagram

flowchart TD
    A[Start] --> B{Check if /data/disable_liveness exists}
    B -- Yes --> C[Output "liveness probe disabled"]
    C --> D[Exit 0]
    B -- No --> E[Send JSON-RPC request to localhost:8545 eth_blockNumber]
    E --> F{Request success?}
    F -- No --> G[Exit 1]
    F -- Yes --> H[Parse JSON result to get current block number (hex)]
    H --> I[Convert hex block number to decimal]
    I --> J{Does /data/.block_number exist?}
    J -- No --> K[Write current block number to /data/.block_number]
    K --> L[Exit 1]
    J -- Yes --> M[Read previous block number from /data/.block_number]
    M --> N[Write current block number to /data/.block_number]
    N --> O{Current block number > previous block number?}
    O -- Yes --> P[Output "daemon is running"]
    P --> Q[Exit 0]
    O -- No --> R[Output "daemon is stalled"]
    R --> S[Exit 1]

Summary

• Purpose: Monitor Ethereum daemon liveness by checking block number progression.

• Type: Utility Bash script, designed as a liveness probe.

• Key Mechanism: Compares current Ethereum block number with last saved number.

• Exit Codes:

  • 0: Healthy/running or disabled.

  • 1: Error or stalled.

• Extensibility: Can be integrated into container orchestration health checks.

• Dependencies: curl, jq, local Ethereum JSON-RPC endpoint.

This documentation should provide a clear understanding of the functionality, operation, and integration of `liveness.sh` within the broader system.