Multi-Blockchain Coinstacks
Overview
The **Multi-Blockchain Coinstacks** module provides modular implementations to support a wide variety of blockchain networks within the ShapeShift Unchained platform. Each blockchain is encapsulated as a distinct "coinstack" comprising several specialized services—primarily the **daemon node(s)**, **indexer service**, and **API servers**—deployed and managed in a Kubernetes environment. This modular design enables scalable, maintainable, and extensible support for diverse blockchains such as Bitcoin, Ethereum, Solana, Thorchain, and their layer 2 or variant networks.
This module addresses the complexity of running heterogeneous blockchain infrastructures by standardizing deployment, configuration, and service orchestration patterns per blockchain. It enables seamless interaction with distinct blockchain protocols via uniform infrastructure and API abstractions.
Core Concepts and Purpose
Coinstack Abstraction: Each blockchain implementation is treated as a coinstack, grouping the necessary components to run the blockchain node, index blockchain data, and expose data and transaction operations via APIs.
Service Modularity: Services within a coinstack are decomposed into:
Daemon Nodes: Full blockchain node clients (e.g., geth for Ethereum, bitcoind for Bitcoin) responsible for node synchronization and blockchain state maintenance.
Indexer Services: Typically Blockbook-based services that efficiently index blockchain data and provide query APIs.
API Servers: REST and WebSocket API endpoints that expose blockchain data and enable client interactions such as querying balances, transaction history, and sending transactions.
Configurability: Each coinstack defines environment-specific configurations, ports, probes, and volumes to tailor to blockchain-specific requirements.
Deployment Automation: Pulumi scripts automate the deployment of each coinstack’s services into Kubernetes, ensuring consistent, repeatable infrastructure provisioning.
Health Management: Liveness, readiness, and startup probes are configured for reliable service monitoring and fault recovery.
How the Multi-Blockchain Coinstacks Module Works
1. Modular Pulumi Deployment Scripts
Each coinstack has a dedicated Pulumi deployment script located under [node/coinstacks/{coin}/pulumi/index.ts](/projects/291/68798) or for Go-based stacks under `go/coinstacks/{coin}/pulumi/index.ts`. These scripts:
Read environment and configuration files (e.g.,
sample.env,config.json).Fetch Kubernetes context and namespace info via a shared
getConfig()utility.Map configured services from the coinstack's
statefulServicesection, creating customized CoinServiceArgs objects for each service.Invoke the shared
deployCoinstack()function from the Pulumi core library, passing prepared arguments to create Kubernetes StatefulSets, Services, ConfigMaps, and IngressRoutes.
For example, the Litecoin coinstack defines two core services, daemon and indexer, with specific ports and probes:
const coinServiceArgs = config.statefulService?.services?.map((service): CoinServiceArgs => {
switch (service.name) {
case 'daemon':
return {
...service,
ports: { 'daemon-rpc': { port: 8332 } },
readinessProbe: { periodSeconds: 30, failureThreshold: 10 },
}
case 'indexer':
return {
...service,
...defaultBlockbookServiceArgs,
configMapData: { 'indexer-config.json': readFileSync('../indexer/config.json').toString() },
}
default:
throw new Error(`no support for coin service: ${service.name}`)
}
})
This pattern is repeated with variations per blockchain, reflecting differences in port assignments, environment variables, volume mounts, and probe configurations.
2. Service Types and Characteristics
Daemon Nodes:
Run blockchain full node clients.
Expose RPC and sometimes WebSocket endpoints.
Include custom environment variables such as snapshots URLs or network identifiers.
Mount configuration files and helper scripts (e.g.,
jwt.hex,evm.sh) as needed.Configured with Kubernetes health probes to monitor node synchronization and responsiveness.
Indexer Services:
Mostly based on Blockbook implementations for UTXO chains and EVM-compatible blockchains.
Read indexer-specific configuration files (
indexer-config.json).Expose HTTP APIs for fast blockchain data queries.
Health probes monitor indexer readiness and liveness.
Additional Services:
Some coinstacks include specialized services, for example:
The Polygon coinstack includes a
heimdallservice for consensus layer interactions.Ethereum and Optimism include beacon node daemons for consensus layers.
Thorchain coinstacks include TimescaleDB services for historical data storage.
3. Deployment Workflow
The coinstack Pulumi script is executed, which:
Loads necessary configuration and environment variables.
Constructs the list of services and their deployment parameters.
Calls
deployCoinstack()which provisions Kubernetes resources using the Pulumi SDK:StatefulSets for each service with specified container images, volumes, ports, and probes.
Services for network routing within the cluster.
ConfigMaps for injecting configuration files into pods.
IngressRoutes for external traffic routing via Traefik.
Kubernetes manages the lifecycle of pods, continuously monitoring health probes to maintain service availability.
Interaction with Other System Components
Infrastructure Layer: The coinstack Pulumi scripts rely on shared Pulumi libraries (
pulumi/src/coinstack) for deployment primitives and Kubernetes resource abstractions.Daemon and Indexer Layers: Daemon services synchronize blockchain state, while indexers consume blockchain data from daemons to build queryable caches.
API Servers: While API servers are typically separate Node.js or Go services interacting with daemons and indexers, the coinstack deployment scripts focus on the underlying node and indexer services. API servers are related but managed in their own deployment scripts elsewhere.
Health and Monitoring: Health and readiness probes configured within coinstack services tie closely with Kubernetes and monitoring tools like Prometheus to provide observability and alerting.
Shared Utilities: Scripts such as
evm.shortendermint.sh, mounted as ConfigMaps, provide lifecycle management for blockchain-specific node clients.Snapshots and State: Some coinstacks configure snapshot URLs via environment variables to bootstrap node synchronization faster.
Important Concepts and Design Patterns
Service Abstraction via CoinServiceArgs: The use of a unified argument structure to describe service deployments abstracts away blockchain-specific details while allowing customization.
Configuration-as-Code: Using Pulumi to define infrastructure declaratively ensures reproducibility and version control of blockchain environments.
Health Probes for Resilience: Startup, readiness, and liveness probes ensure the Kubernetes control plane can detect and respond to node or indexer failures promptly.
Volume Mounts for Config and Scripts: Separation of configuration from container images using ConfigMaps allows dynamic updates and environment-specific overrides.
Modular Extensibility: New blockchains can be added by defining new coinstack directories with their own Pulumi deployment scripts following the established pattern.
Shared Blockbook Defaults: Many UTXO-based chains use common default arguments for their indexer services, encapsulated in
defaultBlockbookServiceArgs.
Code Structure and Examples
The typical coinstack Pulumi entrypoint imports utilities, reads configuration files, and maps services:
import { readFileSync } from 'fs'
import { deployCoinstack } from '../../../../pulumi/src/coinstack'
import { Outputs, CoinServiceArgs, getConfig } from '../../../../pulumi/src'
import { defaultBlockbookServiceArgs } from '../../../packages/blockbook/src/constants'
export = async (): Promise<Outputs> => {
const appName = 'unchained'
const coinstack = 'litecoin'
const sampleEnv = readFileSync('../sample.env')
const { kubeconfig, config, namespace } = await getConfig()
const coinServiceArgs = config.statefulService?.services?.map((service): CoinServiceArgs => {
switch (service.name) {
case 'daemon':
return {
...service,
ports: { 'daemon-rpc': { port: 8332 } },
readinessProbe: { periodSeconds: 30, failureThreshold: 10 },
}
case 'indexer':
return {
...service,
...defaultBlockbookServiceArgs,
configMapData: { 'indexer-config.json': readFileSync('../indexer/config.json').toString() },
}
default:
throw new Error(`no support for coin service: ${service.name}`)
}
})
return deployCoinstack({
appName,
coinServiceArgs,
coinstack,
coinstackType: 'node',
config,
kubeconfig,
namespace,
sampleEnv,
})
}
Variations per blockchain include additional services, environment variables, port mappings, and volume mounts, e.g., Ethereum’s multiple RPC ports and beacon node:
case 'daemon':
return {
...service,
ports: {
'daemon-rpc': { port: 8545 },
'daemon-ws': { port: 8546, pathPrefix: '/websocket', stripPathPrefix: true },
'daemon-auth': { port: 8551, ingressRoute: false },
'daemon-beacon': { port: 3500, ingressRoute: false },
},
configMapData: {
'jwt.hex': readFileSync('../daemon/jwt.hex').toString(),
'evm.sh': readFileSync('../../../scripts/evm.sh').toString(),
},
volumeMounts: [
{ name: 'config-map', mountPath: '/jwt.hex', subPath: 'jwt.hex' },
{ name: 'config-map', mountPath: '/evm.sh', subPath: 'evm.sh' },
],
env: {
NETWORK: config.network,
},
startupProbe: { periodSeconds: 30, failureThreshold: 60, timeoutSeconds: 10 },
livenessProbe: { periodSeconds: 30, failureThreshold: 5, timeoutSeconds: 10 },
readinessProbe: { periodSeconds: 30, failureThreshold: 10, timeoutSeconds: 10 },
}
Mermaid Diagram: Flowchart of Multi-Blockchain Coinstack Deployment Workflow
flowchart TD
A[Start Deployment] --> B[Load Environment & Config Files]
B --> C[Parse Stateful Services from Config]
C --> D[Map Each Service to Deployment Args]
D --> E{Service Type?}
E -->|daemon| F[Configure Daemon Node: Ports, Env, Probes, Volumes]
E -->|indexer| G[Configure Indexer: ConfigMap, Ports, Probes]
E -->|other| H[Configure Additional Services (e.g., Beacon, Heimdall)]
F --> I[Aggregate All Service Args]
G --> I
H --> I
I --> J[Call deployCoinstack() with Args]
J --> K[Provision Kubernetes Resources (StatefulSets, Services, ConfigMaps)]
K --> L[Kubernetes Starts Pods and Runs Probes]
L --> M[Daemon Nodes Synchronize Blockchain]
L --> N[Indexers Build Blockchain Data Cache]
M & N --> O[Services Ready for API Layer Interaction]
O --> P[End Deployment]
This documentation explains the modular architecture of the Multi-Blockchain Coinstacks module, its deployment workflow using Pulumi, and how it organizes blockchain node daemons, indexers, and supporting services into manageable Kubernetes deployments for multi-chain support.