Developer Tooling

This module provides essential tools and configurations designed to facilitate local development, testing, and performance benchmarking of the ShapeShift Unchained platform. It ensures developers can efficiently run and debug blockchain coinstack services locally, simulate realistic workloads, and maintain productive development workflows. The tooling spans container orchestration setups, live code reloading, and load testing scripts.

Docker Compose Environments

Purpose and Overview

The Docker Compose setups automate launching multiple interdependent services locally, replicating key components of the production environment in a lightweight, developer-friendly manner. This eliminates the need for a full Kubernetes cluster during development, enabling faster iteration cycles.

These Compose files primarily define containers for:

• Blockchain node daemons (local nodes)

• API servers for various coinstacks

• Supporting tooling like watchers for hot-reloading

• Reverse proxy for routing and exposing services

Core Docker Compose Configurations

Root docker-compose.yml

Located at the project root, this Compose file defines fundamental development services essential for local environment bootstrapping:

• unchained-local-node & unchained-local-go: Images built from local Dockerfiles (Dockerfile.local) in ./node and ./go/build directories respectively. These represent the core Node.js and Go runtime environments used by coinstacks.

• watcher: A container that runs a yarn-based watcher with nodemon to monitor file changes and trigger automatic recompilation or restarts of Node.js services. It mounts the project source and executes parallel watch commands scoped to relevant packages, ensuring efficient live development feedback.

• reverse-proxy: Runs Traefik v2.5 as a reverse proxy to route HTTP and WebSocket traffic to the appropriate local services based on Docker labels and network configurations. It also exposes a web UI for monitoring traffic and configuration.

The Compose file defines numerous Docker networks corresponding to individual coinstacks (e.g., bitcoin_default, ethereum_default, litecoin_default), allowing network isolation and service discovery within each blockchain's local environment.

Example excerpt from root `docker-compose.yml` showing the watcher service:

services:
  watcher:
    build:
      context: ./node
      dockerfile: Dockerfile.local
    working_dir: /app/node
    command: sh -c "yarn install && yarn lerna run watch --scope @shapeshiftoss/* --parallel"
    volumes:
      - ./:/app

Coinstack-Specific Compose Files

Each blockchain coinstack provides its own Compose configuration, tailored to run its API service and connect to the local node/indexer containers.

For example, the Litecoin coinstack defines a minimal Compose file (`node/coinstacks/litecoin/docker-compose.yml`) to run its API container with appropriate Traefik labels and network configuration:

services:
  api:
    image: unchained-local-node
    env_file: .env
    labels:
      - 'traefik.enable=true'
      - 'traefik.http.routers.litecoin-api.rule=Host(`api.litecoin.localhost`)'
      - 'traefik.http.services.litecoin-api.loadbalancer.server.port=3000'
    working_dir: /app/node/coinstacks/litecoin/api
    command: yarn nodemon
    volumes:
      - ../../..:/app
    networks:
      - litecoin

networks:
  litecoin:
    name: litecoin_default
    external: true

This setup mounts the source code into the container enabling hot reload via `nodemon`, and configures Traefik to route requests on `api.litecoin.localhost` to port 3000 of this container.

Interaction with Other Modules

• The Compose environments leverage the local versions of blockchain daemons and API servers implemented in the node/coinstacks/* directories.

• Traefik reverse proxy integrates with these containers to unify routing for multiple blockchains.

• Watcher service uses lerna to monitor monorepo packages referenced by the API servers.

• Developers can simulate the full API layer and blockchain interaction locally without external dependencies.

Performance Testing

Purpose and Overview

Performance testing scripts provide automated, repeatable load testing scenarios to benchmark the API services under realistic traffic conditions. These tests validate system scalability, response latency, and error rates, identifying bottlenecks and ensuring reliability under heavy client usage.

Key Tools and Scripts

Performance tests are implemented using the k6 load testing tool, with scripts organized by blockchain under the `k6/` directory.

Ethereum Load Test (k6/ethereum/ethereum_test.js)

• Executes high volume GET requests against Ethereum account endpoints.

• Uses a constant arrival rate executor simulating 1000 requests per second over 5 minutes.

• Requests are distributed over a large list of Ethereum addresses to mimic diverse client queries.

• Specifies thresholds such as 95th percentile latency under 800ms and error rate below 1%.

Example scenario configuration snippet:

export const options = {
	scenarios: {
		rps: {
			executor: 'constant-arrival-rate',
			duration: "5m",
			rate: 1000,
			preAllocatedVUs: 2500,
			maxVUs: 10000,
		},
	},
	thresholds: {
		'http_req_duration{scenario:rps}': ['p(95) < 800'],
		'http_req_failed{scenario:rps}': ['rate<0.01'],
	},
}

Bitcoin Load Test (k6/bitcoin/bitcoin_test.js)

• Simulates load on Bitcoin account-related endpoints, including both account info and unspent transaction outputs (UTXOs).

• Defines separate scenarios for account data and UTXO data with similar constant arrival rates.

• Targets endpoints using a set of extended public keys (xpubs).

• Enforces latency and failure rate thresholds comparable to Ethereum tests.

Example execution function for UTXOs:

export function utxos() {
	http.get(`https://api.bitcoin.shapeshift.com/api/v1/account/${xpubs[exec.scenario.iterationInTest % xpubs.length]}/utxos`)
}

Interaction with Other Modules

• These scripts target the running API servers deployed locally or in test environments.

• The load tests help validate the API’s connection and querying of underlying indexers and daemon nodes.

• Results support tuning and scaling decisions in deployment automation modules.

• Summary output files (result.json) facilitate integration with CI pipelines and performance dashboards.

Development Workflow Integration

• Developers can spin up local blockchain environments using the root and coinstack-specific Docker Compose files to run API servers and nodes with live reload.

• The watcher service ensures code changes propagate immediately to running containers, improving feedback loops.

• Traefik proxy centralizes request routing, enabling simple access to multiple blockchain APIs via hostname-based rules.

• Performance testing scripts provide a standard method to evaluate service health and readiness before merging code or releasing.

Visual Diagram: Developer Tooling Workflow

flowchart TD
  Dev[Developer] -->|Edit Code| FS[Local File System]
  FS -->|Changes Detected| Watcher[Watcher Service (nodemon + lerna)]
  Watcher -->|Rebuild & Restart| API[API Containers per Coinstack]
  API -->|Serve Requests| Client[Local or Remote Clients]
  Dev -->|Start Services| DockerCompose[Docker Compose]
  DockerCompose --> API
  DockerCompose --> Node[Local Blockchain Daemons]
  DockerCompose --> Proxy[Traefik Reverse Proxy]
  Client -->|HTTP/WS Requests| Proxy
  Proxy --> API
  Dev -->|Run Load Tests| K6[k6 Performance Scripts]
  K6 -->|HTTP Requests| Proxy

This flowchart illustrates how developers interact with the tooling: editing source files triggers watcher service recompilation and API container reload, while Docker Compose orchestrates running services behind Traefik. Load tests run via k6 generate traffic routed through the proxy to evaluate performance.

This detailed explanation covers the purpose, configurations, workflows, and interactions of the Developer Tooling module, emphasizing local environment setup and performance benchmarking for efficient development and testing of multi-blockchain APIs.