compose.j2

Overview

This file is a Jinja2 template for a Docker Compose configuration that defines multiple services and their runtime configurations for a containerized environment. It leverages templating variables and conditional logic to dynamically generate a docker-compose.yaml file tailored to deployment needs. The primary purpose is to orchestrate the deployment of several interdependent services including a web server, GraphQL API server, block manager, staking service, and log rotation utility within a defined network.

Services Description

nginx_bm

• Purpose: Acts as the NGINX web server container, likely serving as a reverse proxy or load balancer.

• Image: Uses the image defined by the NGINX_IMAGE variable.

• Restart Policy: Always restarts unless explicitly stopped.

• Networks: Connected to the ackinacki-net network.

• Environment Variables:

  • MESSAGE_ROUTER_ADDRESS: Points to the block manager API.

  • Q_SERVER_ADDRESS: Points to the GraphQL server (q_server_bm).

  • NODE_API_ADDRESS and BLOCK_MANAGER_API: Both point to the block manager API.

• Ports Configuration:

  • Publishes port 80 inside the container to a host IP and port defined via templating logic.

  • The host IP is conditionally selected based on NGINX_IP:

    • If "public": uses HOST_PUBLIC_IP

    • If "private": uses HOST_PRIVATE_IP

    • Otherwise: uses the value of NGINX_IP directly.

  • The published port is defined by NGINX_PORT.

q_server_bm

• Purpose: Runs a GraphQL server that interfaces with the block manager.

• Image: Defined by GQL_IMAGE.

• Entrypoint: Overridden to an empty list to allow custom command execution.

• Restart Policy: Unless stopped.

• Environment Variables:

  • NODE_VERBOSE: Set to 1 to enable verbose logging.

  • BM_API_SOCKET: Socket address for block manager communication.

• Command: Runs a bash shell that launches the GraphQL server (gql-server) serving a SQLite database (bm-archive.db) on port 3000.

• Expose: Internally exposes port 3000.

• Ports: Maps port 3000 inside the container to a host private IP and port.

• Volumes: Mounts a persistent data directory from the host.

• Networks: Connected to ackinacki-net.

• Dependencies: Depends on the block_manager service to ensure correct startup order.

block_manager

• Purpose: Core service managing blocks, likely related to blockchain or distributed ledger functionality.

• Image: Defined by BM_IMAGE.

• Entrypoint: Overridden to an empty list.

• Restart Policy: Unless stopped.

• Init: Enabled to ensure proper signal handling and process reaping.

• Environment Variables:

  • Include debugging and logging flags (RUST_BACKTRACE, RUST_LOG).

  • Various URLs and ports for HTTP, streaming, API, and REST endpoints.

  • Wallet and key file paths.

  • Token TTL and API tokens.

• Command: Executes a migration tool for database setup, copies database schema, and then launches the block manager binary with logging redirected.

• Expose: Exposes API port internally.

• Ports:

  • Binds and publishes API and REST API ports with specified host addresses.

• Volumes:

  • Mounts directories for data, configs, and logs.

• Networks: Connected to ackinacki-net.

• Includes: Incorporates additional environment variables from an external Jinja2 snippet (extra/block-manager-env.j2), if available.

staking_bm

• Purpose: Runs a staking service related to the block manager, potentially handling validator or staking operations.

• Image: Defined by STAKING_BM_IMAGE.

• Restart Policy: Unless stopped.

• Entrypoint: Empty list.

• Command:

  • Configures a TVM CLI client with a URL endpoint.

  • Adjusts script permissions.

  • Executes a staking shell script with parameters for staking time and wallet keys.

• Volumes: Mounts config directory.

• Dependencies: Depends on block_manager.

• Networks: Connected to ackinacki-net.

logrotate

• Purpose: Manages log rotation for the block manager logs.

• Image: Defined by LOGROTATE_IMAGE.

• Restart Policy: Unless stopped.

• Environment Variables: Defines a cron schedule for log rotation.

• Volumes: Mounts log directory and a logrotate shell script.

• Command: Executes the logrotate script.

• Init: Enabled for proper initialization.

Networks

• ackinacki-net:

  • Defines a named network ackinacki-net.

  • Optionally marked as external if EXTERNAL_NET is set.

Implementation Details and Algorithms

• Templating Logic: Uses Jinja2 conditional statements to select host IP addresses dynamically for the nginx_bm service port mapping. This adds flexibility to deploy the same template in different networking environments (public, private, or custom IP).

• Service Dependencies: q_server_bm and staking_bm explicitly depend on the block_manager service to ensure proper startup sequencing, avoiding race conditions or service unavailability during initialization.

• Command Chaining: The block_manager service uses a chained bash command to perform database migrations and schema setup before launching the main binary, ensuring the environment is prepared before service startup.

• Volume Mounts: Persistent storage is configured for data, configurations, and logs to maintain state across container restarts and support persistent data integrity.

• Network Isolation: All services are connected to a dedicated Docker network ackinacki-net to facilitate secure and efficient inter-service communication.

Interactions with Other System Components

• Block Manager API: Multiple services reference the block manager API endpoints (block_manager:BM_API_PORT) for communication, showing that the block manager is central to the system's operations.

• GraphQL Server (q_server_bm): Acts as an interface layer exposing block manager data via GraphQL, suggesting integration with frontend clients or other API consumers.

• NGINX Server: Serves as a gateway proxy to route requests to internal services (block_manager, q_server_bm), managing incoming traffic and possibly handling SSL termination or load balancing.

• Staking Service: Interfaces with the block manager and external TVM (possibly a blockchain virtual machine or smart contract environment) endpoint, indicating a role in staking or consensus participation.

• Logrotate Service: Manages the lifecycle of logs generated by the block manager service, ensuring log files do not grow indefinitely and system stability is maintained.

Usage Examples

• Starting Services: After rendering the template with appropriate variables, use docker-compose up -d to start all services in detached mode.

• Customizing Ports and IPs: Modify NGINX_IP, NGINX_PORT, and related variables to adjust which host IP and ports are bound for external access.

• Adding Environment Variables: Extend or override environment configurations by editing the extra/block-manager-env.j2 snippet included in the block manager service.

Visual Diagram of Service Structure

flowchart LR
nginx_bm["nginx_bm (NGINX)"]
q_server_bm["q_server_bm (GraphQL Server)"]
block_manager["block_manager (Block Manager)"]
staking_bm["staking_bm (Staking)"]
logrotate["logrotate (Log Rotation)"]
ackinacki_net["ackinacki-net Network"]
nginx_bm -->|API calls| block_manager
nginx_bm -->|API calls| q_server_bm
q_server_bm -->|Depends on| block_manager
staking_bm -->|Depends on| block_manager
block_manager -.->|Logs stored| logrotate
nginx_bm --- ackinacki_net
q_server_bm --- ackinacki_net
block_manager --- ackinacki_net
staking_bm --- ackinacki_net
logrotate --- ackinacki_net

This diagram illustrates the interactions between services and their shared network, emphasizing dependency and communication flows.

Detailed Explanation of Key Configuration Elements

Environment Variables in block_manager

• RUST_BACKTRACE and RUST_LOG: Enable detailed Rust error backtraces and debug-level logging, useful for diagnosing runtime issues.

• HTTP_SRC_URL, STREAM_SRC_URL: Define source URLs for HTTP and streaming endpoints the block manager interacts with.

• BLOCK_MANAGER_API, REST_API: Define listening addresses for the block manager's API and REST interfaces.

• Wallet and key-related variables (BM_OWNER_WALLET_PUBKEY, BM_ISSUER_KEYS_FILE): Point to cryptographic credentials for signing and authentication.

• TOKEN_TTL and BK_API_TOKEN: Control token expiration and API authorization tokens for secure inter-service communication.

Conditional Port Binding in nginx_bm

The templated conditional block:

{% if NGINX_IP == "public" %}
  host_ip: {{ HOST_PUBLIC_IP }}
{% elif NGINX_IP == "private" %}
  host_ip: {{ HOST_PRIVATE_IP }}
{% else %}
  host_ip: {{ NGINX_IP }}
{% endif %}

determines the host IP address for NGINX port publishing, enabling environment-specific deployments without modifying the Compose file manually.

Inclusion of External Environment Variables

The line:

{% include 'extra/block-manager-env.j2' ignore missing +%}

allows injecting additional environment variables or configurations into the block manager service from an external template file if present, providing extensibility.

Volume Mounting Strategy

• Data directories are mounted inside containers to /workdir/data or /bm-configs, ensuring persistence of database files, wallet keys, and configuration files.

• Log directories are mounted for centralized logging and rotation.

Relations to Other Topics

• Service orchestration and container networking concepts are related to Container Orchestration.

• Environment variable templating and configuration management are connected to Configuration Management.

• Service dependencies and startup order relate to Service Dependency Management.

• Database migration and initialization techniques correspond with Database Migration.

• Log management and rotation are detailed in Log Management.