Metrics Collection

Purpose

Metrics Collection focuses on capturing detailed runtime statistics from blockchain API servers and related services to enable observability and performance monitoring. Unlike the broader monitoring topic that includes alerting and dashboarding, this subtopic specifically addresses gathering, structuring, and exposing Prometheus metrics for HTTP request patterns, WebSocket client connections, and Kubernetes resource states within the blockchain coinstack infrastructure.

The goal is to provide real-time insight into API usage, latency, and connection health, which helps operators optimize performance, detect anomalies, and maintain system reliability.

Functionality

This subtopic implements a Prometheus metrics client integrated into the API server stack. Key workflows and features include:

• HTTP Request Metrics: Counting the number of HTTP requests, categorized by HTTP method, route, and response status code. Additionally, it measures the duration of each HTTP request to analyze latency patterns.

• WebSocket Connection Metrics: Tracking the number of active WebSocket client connections to the API server, providing visibility into real-time event subscription load.

• Default System Metrics: Automatically collecting standard Node.js process and runtime metrics (CPU, memory, garbage collection, etc.) via Prometheus’s default collectors.

• Scoped Registration: Metrics are registered with labels that identify the specific blockchain coinstack (e.g., Bitcoin, Ethereum), enabling filtering and aggregation per blockchain service instance.

• Metric Exposure: Metrics are exposed via an HTTP endpoint for Prometheus to scrape periodically, supporting integration with Kubernetes monitoring.

Example: Prometheus Metrics Setup Class

The following snippet from [prometheus.ts](/projects/291/69099) illustrates the core class encapsulating Prometheus metric registration:

import client from 'prom-client'

export class Prometheus {
  register: client.Registry

  metrics = {
    httpRequestCounter: new client.Counter({
      name: 'unchained_http_request_count',
      help: 'Count of http requests',
      labelNames: ['method', 'route', 'statusCode'],
    }),
    httpRequestDurationSeconds: new client.Histogram({
      name: 'unchained_http_request_duration_seconds',
      help: 'Duration of HTTP requests in seconds',
      labelNames: ['method', 'route', 'statusCode'],
    }),
    websocketCount: new client.Gauge({
      name: 'unchained_ws_client_count',
      help: 'Count of websocket client connections',
    }),
  }

  constructor({ coinstack }: { coinstack: string }) {
    this.register = new client.Registry()
    this.register.setDefaultLabels({ coinstack })

    client.collectDefaultMetrics({ register: this.register })

    Object.values(this.metrics).forEach((value) => this.register.registerMetric(value))
  }
}

• httpRequestCounter and httpRequestDurationSeconds capture request counts and latencies labeled by HTTP method, route, and status code.

• websocketCount tracks live WebSocket client connections.

• Default process metrics such as CPU and memory usage are collected by client.collectDefaultMetrics.

• The coinstack label allows metrics to be differentiated by blockchain service.

Integration

Metrics Collection integrates tightly with the Unified API Layer and Kubernetes infrastructure:

• Within the API Servers: The Prometheus metrics client is instantiated per blockchain coinstack API server, embedding metrics instrumentation directly into HTTP request handlers and WebSocket connection management. This enables precise measurement of API traffic and client subscriptions.

• With Kubernetes Monitoring: Metrics endpoints exposed by API servers are scraped by Prometheus instances deployed in the Kubernetes cluster. These Prometheus instances and related dashboards are configured via Deployment Automation scripts (e.g., Helm charts deployed through Pulumi).

• Complementary to Alerting and Visualization: The collected metrics feed into the Alerting Configuration subtopic, enabling alert rules based on thresholds (e.g., high latency or connection drops), and are visualized through Grafana dashboards.

• Supports Multi-Blockchain Monitoring: By labeling metrics with the coinstack identifier, operators can monitor multiple blockchain services side-by-side, facilitating cross-chain performance analysis.

This subtopic introduces the standardized, code-level setup for metrics registration and collection, which is not covered by the broader monitoring topic or alerting configuration. It provides the foundation upon which monitoring dashboards and alerting rules are built.

Diagram

flowchart TD
  API[API Server] -->|HTTP Request| Middleware[Prometheus Middleware]
  Middleware -->|Increment Counters & Observe Latency| MetricsRegistry[Prometheus Registry]
  API -->|WebSocket Client Connect/Disconnect| WSHandler[WebSocket Connection Handler]
  WSHandler -->|Update Gauge| MetricsRegistry
  MetricsRegistry -->|Expose Metrics| /metrics[HTTP /metrics Endpoint]
  PrometheusServer[Prometheus Scraper] -->|Scrape /metrics| /metrics
  PrometheusServer -->|Store Metrics Data| TSDB[(Time-Series DB)]
  TSDB -->|Visualize| Grafana[Grafana Dashboard]

• Incoming HTTP requests pass through middleware that increments counters and records durations.

• WebSocket connection events update gauge metrics.

• Metrics registry exposes data at the /metrics endpoint.

• Prometheus periodically scrapes the exposed metrics and stores them.

• Grafana visualizes collected metrics for monitoring and alerting purposes.

This Metrics Collection subtopic establishes a robust and extensible framework for gathering operational metrics, enabling effective observability and proactive system management across diverse blockchain coinstacks.