Kubernetes Resource Management

Purpose

This subtopic addresses the automated definition and deployment of Kubernetes resources essential for running blockchain node services, indexers, and APIs within the ShapeShift Unchained platform. It specifically focuses on managing StatefulSets, Services, ConfigMaps, and IngressRoutes, which are critical Kubernetes constructs enabling reliable, scalable, and maintainable operation of blockchain coinstack components.

In the broader context of deployment automation, Kubernetes Resource Management ensures that each blockchain service is correctly containerized, configured, networked, and exposed with appropriate lifecycle management. It solves the problem of consistently and declaratively provisioning these resources across different blockchain coinstacks, environments, and infrastructure providers.

Functionality

At the core of this subtopic is a Pulumi-based TypeScript module that programmatically creates and deploys Kubernetes resource manifests tailored to each blockchain service’s requirements.

Key workflows and methods include:

• Service Definition (createCoinService):
  Builds detailed service specifications including container images, environment variables, ports, resource limits, and crucially, lifecycle probe scripts (init.sh, startup.sh, liveness.sh, and readiness.sh). These scripts are read from the respective coinstack directories to embed health checks and startup logic into the Kubernetes pods.

• Pod Configuration:
  Containers are configured with volume mounts for persistent data storage and for mounting ConfigMaps containing the lifecycle scripts. Optional monitor containers may be added to handle readiness probes separately, improving pod health management.

• Persistent Storage (volumeClaimTemplates):
  Defines PersistentVolumeClaims with configurable storage classes and optional AWS EBS-specific performance annotations (IOPS and throughput). This ensures blockchain daemons have durable and performant storage for their data directories.

• StatefulSet Deployment (deployStatefulService):
  Aggregates multiple service definitions and deploys them as a single StatefulSet, ensuring ordered, stable, and persistent pods. It also creates the necessary Kubernetes Service objects for internal networking.

• ConfigMap Management:
  Consolidates lifecycle scripts and additional configuration data into ConfigMaps mounted inside pods, allowing for dynamic and version-controlled configuration without rebuilding container images.

• Ingress and TLS Setup:
  When a root domain is configured, this subtopic automates the creation of TLS certificates via cert-manager resources and sets up Traefik IngressRoutes and Middleware. This enables secure external access to the services with hostname-based routing and path prefix stripping when needed.

• Labeling and Selector Strategy:
  Consistent labeling of Kubernetes objects (app, asset, tier) facilitates effective service discovery, monitoring, and management within the cluster.

Example Code Snippet: Creating a Coin Service Definition

const serviceContainer: k8s.types.input.core.v1.Container = {
  name,
  image: args.image,
  command: initScript && !args.command ? ['/init.sh'] : args.command,
  env,
  ports: ports.map(({ port: containerPort, name }) => ({ containerPort, name })),
  startupProbe: startupProbe && { exec: { command: ['/startup.sh'] } },
  livenessProbe: livenessProbe && { exec: { command: ['/liveness.sh'] } },
  readinessProbe: readinessProbe && { exec: { command: ['/readiness.sh'] } },
  volumeMounts: [
    { name: `data-${args.name}`, mountPath: args.dataDir ?? '/data' },
    // Mount lifecycle scripts ConfigMap
    { name: 'config-map', mountPath: '/init.sh', subPath: `${args.name}-init.sh` },
    // ... other lifecycle scripts
  ],
}

Example Code Snippet: Deploying StatefulSet and Services

new k8s.core.v1.Service(
  `${assetName}-svc`,
  {
    metadata: { name: `${assetName}-svc`, namespace, labels },
    spec: { ports, selector: labels, type: 'ClusterIP' },
  },
  { provider, deleteBeforeReplace: true }
)

new k8s.apps.v1.StatefulSet(
  `${assetName}-sts`,
  {
    metadata: { name: `${assetName}-sts`, namespace, labels },
    spec: {
      selector: { matchLabels: labels },
      serviceName: `${assetName}-svc`,
      replicas: config.statefulService.replicas,
      podManagementPolicy: 'Parallel',
      updateStrategy: { type: 'RollingUpdate' },
      template: podSpec,
      volumeClaimTemplates,
    },
  },
  { provider }
)

Integration

Kubernetes Resource Management is a foundational pillar within the Deployment Automation parent topic, responsible for translating high-level blockchain service configurations into concrete Kubernetes resources.

• With Deployment Automation:
  It acts as the executor of deployment plans prepared by Pulumi scripts, handling the nuances of Kubernetes API objects for each blockchain coinstack.

• With Docker Image Automation:
  The container images referenced in the service configurations are produced and tagged by the Docker Image Automation subtopic, ensuring that the deployed pods run the correct software versions.

• With Cluster Setup and IPFS:
  This subtopic assumes an existing Kubernetes cluster (provisioned by the Cluster Setup subtopic) and integrates with IPFS deployments where applicable by mounting volumes or configuring networking.

• With Health and Readiness Probes (via lifecycle scripts):
  The probe shell scripts mounted into pods are authored and maintained by the Infrastructure Scripts subtopic, providing the health check logic executed by Kubernetes.

• With Monitoring and Observability:
  Labels and consistent resource naming allow Prometheus and Grafana monitoring configurations to discover and track these services’ health and performance metrics.

• With Multi-Blockchain Coinstacks:
  This subtopic provides the reusable mechanism to deploy any blockchain coinstack’s daemon, indexer, and API service by passing in coinstack-specific configurations and probe scripts.

By abstracting Kubernetes resource management into programmable constructs, this subtopic enables consistent, repeatable, and scalable deployments across multiple blockchain nodes and environments, greatly reducing manual configuration and operational error.

Diagram

A flowchart illustrating the core process of Kubernetes resource creation and deployment for blockchain services:

flowchart TD
  Start[Start Deployment] --> LoadConfig[Load Coinstack Config & Env]
  LoadConfig --> CreateServices[Call createCoinService for Each Service]
  CreateServices --> AggregateResources[Aggregate Containers, Ports, ConfigMaps]
  AggregateResources --> CreateK8sObjects[Create Service & ConfigMap Resources]
  CreateK8sObjects --> DeployStatefulSet[Deploy StatefulSet with Volumes & Probes]
  DeployStatefulSet --> SetupIngress[Configure TLS Certificates & IngressRoutes]
  SetupIngress --> Complete[Deployment Complete]

This visualization emphasizes the modular process of converting configuration inputs into fully managed Kubernetes resources supporting blockchain service pods with health checks, storage, networking, and secure external access.