server.rs

Overview

This file implements the main server application for a proxy service that integrates gossip protocols, network messaging, and configuration hot reloading. It orchestrates network transport using MsQuic and UDP transports, manages configuration and certificate reloading, handles gossip communication, and multiplexes incoming and outgoing messages. The server also sets up telemetry and metrics collection for observability.

Key functionalities include:

• Parsing command-line arguments for configuration.

• Loading and managing proxy configuration with hot reload support.

• Running gossip protocols to discover and communicate with peers.

• Setting up network pub-sub messaging with a multiplexor bridging incoming and outgoing messages.

• Observability via OpenTelemetry metrics.

• Managing trusted public keys and TLS certificate caching.

• Coordinating asynchronous tasks for gossip, configuration reload, and message multiplexing.

Structs and Types

CliArgs

Command-line arguments for the proxy server, implemented using clap::Parser.

Fields:

Attributes:

• #[command(author, long_version = &**LONG_VERSION, about)] - auto-generated CLI metadata including version info.

• #[arg(short, long, default_value = "config.yaml")] for config.

• #[arg(long, env, default_value_t = 1000)] for max_connections.

Methods:

• async fn run(self, net_metrics: Option<NetMetrics>) -> anyhow::Result<()>

  Runs the server with the provided metrics instance. Loads configuration, initializes transports, spawns tasks, and manages shutdown coordination.

Usage Example:

let args = CliArgs::parse();
args.run(Some(net_metrics_instance)).await?;

NodeId

A simple wrapper struct for node identifiers used in gossip subscriptions.

Fields:

• String: encapsulates the ID string.

Traits Implemented:

• Clone, Debug, PartialEq, Eq, Hash for collection and comparison support.

• Display for formatting (displays the inner string).

• FromStr for parsing from string.

Constants

Static Variables

LONG_VERSION

A lazily initialized string containing detailed build and version information, including git branch, commit, date, and build time.

Functions

run() -> anyhow::Result<()>

Entrypoint function that initializes the environment, installs crypto providers, and creates a Tokio runtime for asynchronous execution. Calls tokio_main() inside the runtime.

• Prints version info and startup messages.

• Loads environment variables using dotenvy.

• Installs the default crypto provider from rustls::crypto::ring.

• Creates and runs Tokio multi-threaded runtime.

tokio_main() -> anyhow::Result<()>

Async main function executed inside Tokio runtime.

• Parses CLI arguments.

• Initializes OpenTelemetry meter provider and metrics (NetMetrics, TokioMetrics).

• Calls CliArgs::run() with metrics.

• Shuts down meter provider on exit.

dispatch_hot_reload(...)

Asynchronously listens for changes in configuration, trusted keys, and shutdown signals. Updates network, gossip, and watch gossip configurations and broadcasts changes downstream via watch channels.

Parameters:

Algorithm:

• Uses dispatch_configs helper to compute current configs.

• Enters loop waiting for changes.

• On change, sends updated configs if they differ.

• Exits on shutdown or channel closure.

dispatch_configs(...) -> Option<(NetworkConfig, GossipConfig, WatchGossipConfig)>

Helper function to produce updated network, gossip, and watch gossip configurations.

Parameters:

Returns:

• Option containing a tuple of updated (NetworkConfig, GossipConfig, WatchGossipConfig).

• Returns None if network configuration loading fails.

Implementation Details:

• Loads network configuration from proxy config.

• Merges trusted public keys from bk_set and network credentials.

• Constructs WatchGossipConfig with a fixed max_nodes_with_same_id of 5.

message_multiplexor(...) -> anyhow::Result<()>

An async task that bridges incoming network messages to outgoing broadcast channels.

Parameters:

Behavior:

• Receives messages asynchronously.

• Logs and updates metrics for incoming and outgoing phases.

• Forwards incoming messages as broadcast messages, excluding the original connection.

• Terminates on channel closure.

CliArgs::run Method Detailed Workflow

1. Loads TLS certificate cache.

2. Loads proxy configuration from file.

3. Initializes multiple Tokio watch channels for config propagation:

   • Proxy config

   • Network config

   • Gossip config

   • Watch gossip config

   • Subscriptions and peers

4. Starts gossip subsystem using UDP transport.

5. Spawns critical tasks:

   • Gossip watcher task (watch_gossip).

   • Configuration reload handler (config_reload_handler).

   • Message multiplexor (message_multiplexor).

   • Network pub-sub handler (network::pub_sub::run).

   • BK set watcher task (bk_set_watcher::run).

   • Hot reload dispatcher.

6. Runs tokio select on spawned tasks to detect any critical failure and exit.

Interactions with Other Modules

• gossip: Runs gossip protocol tasks and manages peer discovery and messaging. Uses UdpTransport for network communication.

• network::pub_sub: Manages pub-sub messaging including incoming/outgoing connection handling and message dispatch.

• network::resolver: Provides configuration watching for gossip subscriptions.

• transport_layer::msquic: Provides QUIC-based transport abstraction.

• transport_layer::TlsCertCache: Caches TLS certificates for secure connections.

• crate::config: Loads and reloads proxy configuration.

• crate::bk_set_watcher: Watches and updates trusted public keys from an external source.

• telemetry_utils: Initializes telemetry metric providers.

• opentelemetry::global: Sets up global meter provider for metrics.

Important Implementation Details

• Uses Tokio async runtime with multi-threaded scheduler for concurrency.

• Uses tokio::sync::watch and tokio::sync::broadcast channels extensively for configuration and message state propagation.

• Metrics are collected at key message delivery phases using NetMetrics and OpenTelemetry.

• Implements dynamic hot-reloading of configurations and trusted key sets without server restart.

• The message multiplexor forwards incoming messages to all subscribers except the original sender.

• The BK set watcher periodically fetches updated sets of trusted public keys to maintain secure peer trust.

• Error handling is done with anyhow::Result to propagate errors with context.

• Uses LazyLock to lazily initialize and cache versioning information.

Visual Diagram

flowchart TD
A["run()"] --> B["tokio_main()"]
B --> C["CliArgs::run()"]
C --> D[Load Config & TLS Cache]
C --> E[Spawn Gossip Tasks]
C --> F[Spawn Config Reload Handler]
C --> G[Spawn Message Multiplexor]
C --> H[Spawn Network PubSub Task]
C --> I[Spawn BK Set Watcher]
C --> J[Spawn Hot Reload Dispatcher]
G --> K[Forward IncomingMessage]
K --> L[Broadcast OutgoingMessage]
J --> M["dispatch_configs()"]
J --> N[Update Config Channels]
style A fill:#f9f,stroke:#333,stroke-width:1px
style B fill:#ccf,stroke:#333,stroke-width:1px
style C fill:#cfc,stroke:#333,stroke-width:1px
style G fill:#fcf,stroke:#333,stroke-width:1px
style H fill:#cff,stroke:#333,stroke-width:1px
style J fill:#ffc,stroke:#333,stroke-width:1px

Usage Example of Server Launch

fn main() -> anyhow::Result<()> {
    server::run()
}

This will start the server process, parse command-line arguments, load configuration, and run all necessary background tasks.

References to Relevant Topics

• Configuration management and hot reload handled in ProxyConfig and related config modules Configuration Management.

• Gossip protocol and peer discovery are detailed in Gossip Protocol.

• Network pub-sub messaging and multiplexing concepts can be found in Pub-Sub Messaging.

• Transport abstraction including MsQuic and UDP transport layers are covered in Network Transport.

• Observability and metrics integration using OpenTelemetry is described in Telemetry & Metrics.

• TLS certificate caching and management is explained in TLS Security.