gossip.rs

Overview

The gossip.rs file implements a networked gossip protocol node using the chitchat library. It sets up a node capable of joining a cluster, exchanging state information with peers, and exposing a REST API to inspect and modify the local node state. The node periodically gossips with other nodes to maintain a consistent cluster state. This file provides the entry point of the application, configuration parsing, initialization of the gossip node, and a minimal HTTP API for interacting with the node's key-value state.

Components

Structs and Types

Api

• Purpose: Implements the REST API endpoints exposed by the node.

• Fields:

  • chitchat: ChitchatRef — a thread-safe reference to the running chitchat node instance.

Opt

• Purpose: Command-line options parsed using structopt to configure the gossip node and server.

• Fields:

  • listen_addr: SocketAddr — the socket address where the node listens for gossip messages and HTTP API requests.

  • public_addr: Option<SocketAddr> — optional public address advertised to other nodes (defaults to listen_addr).

  • node_id: Option<String> — unique node identifier; if not provided, generated automatically.

  • seeds: Vec<String> — list of seed nodes to join the cluster.

  • interval: u64 — gossip interval in milliseconds.

Functions and Methods

Api::index

async fn index(&self) -> Json<serde_json::Value>

• Description: HTTP GET endpoint at / that returns the current gossip cluster state.

• Returns: JSON object containing:

  • cluster_id: the cluster identifier string.

  • cluster_state: a snapshot of the cluster's state.

  • live_nodes: list of live nodes currently connected.

  • dead_nodes: list of nodes considered dead by the failure detector.

• Usage Example:

curl http://<node_address>/

• Implementation Detail: Locks the chitchat instance to read state atomically.

Api::set_kv

async fn set_kv(&self, key: Query<String>, value: Query<String>) -> Json<serde_json::Value>

• Description: HTTP GET endpoint at /set_kv/ that sets a key-value pair in the node's local state without validation.

• Parameters:

  • key: the key to set in the local state (query parameter).

  • value: the corresponding value to set (query parameter).

• Returns: JSON indicating success status { "status": true }.

• Usage Example:

curl "http://<node_address>/set_kv/?key=mykey&value=myvalue"

• Implementation Detail: Uses the self_node_state() API from chitchat to mutate the local node state.

generate_server_id

fn generate_server_id(public_addr: SocketAddr) -> String

• Purpose: Generates a unique server/node ID string by combining the public address with a randomly generated suffix.

• Parameters:

  • public_addr: socket address of the node.

• Returns: A string in the format "server:<public_addr>-<random_id>".

• Implementation Detail: Uses cool_id_generator crate to produce a medium size ID.

main

#[tokio::main]
async fn main() -> anyhow::Result<()>

• Purpose: Entry point of the program; initializes and runs the gossip node and HTTP server.

• Steps:

  1. Initializes tracing/logging.

  2. Parses CLI arguments into an Opt instance.

  3. Determines the public address and node ID.

  4. Creates a ChitchatId with node ID, generation (based on Unix timestamp), and public address.

  5. Configures ChitchatConfig with cluster ID, gossip interval, seed nodes, and failure detector settings.

  6. Spawns the gossip node using spawn_chitchat with UDP transport.

  7. Sets an initial key-value "bind" to the public address in the node's local state.

  8. Creates and serves an OpenAPI HTTP API on the provided listen address.

  9. Spawns a background Tokio task that logs live nodes and their bind addresses every second.

  10. Runs the HTTP server until termination.

• Return: anyhow::Result<()> indicating success or failure.

• Implementation Details:

  • Uses the chitchat crate for gossip protocol functionality.

  • Uses the poem and poem_openapi crates for HTTP server and REST API.

  • Uses Tokio for asynchronous runtime.

Implementation Details and Algorithms

• Gossip Protocol: Built on top of the chitchat library, which implements a scalable, failure-detecting gossip protocol for cluster membership and state dissemination.

• Failure Detection: Configured with default failure detector parameters via FailureDetectorConfig::default().

• State Management: Each node maintains a key-value map representing its state (self_node_state). Other nodes receive updates through gossip.

• Cluster Membership: Nodes join based on seed addresses provided in the CLI.

• API Concurrency: The chitchat instance is wrapped in a lock to ensure thread-safe mutation and state retrieval.

• Periodic Logging: A Tokio task logs live node IDs and their bind addresses every second for monitoring purposes.

• Node ID Generation: If not specified, a unique node ID is generated combining the public address and a random suffix to avoid collisions.

Interaction with Other System Components

• chitchat Library: Core dependency providing the gossip protocol mechanics, node state management, and failure detection.

• cool_id_generator Crate: Generates unique IDs used for node identification.

• HTTP Server and API: Exposes node status and allows key-value updates via RESTful endpoints using poem and poem_openapi.

• CLI Interface: Uses structopt for parsing command-line options to configure node behavior.

• Network Transport: Uses UDP via chitchat::transport::UdpTransport for gossip message exchange.

• Tokio Runtime: Manages asynchronous execution of the gossip protocol, API server, and background tasks.

Visual Diagram

classDiagram
class Api {
+chitchat: ChitchatRef
+index()
+set_kv()
}
class Opt {
+listen_addr: SocketAddr
+public_addr: Option<SocketAddr>
+node_id: Option<String>
+seeds: Vec<String>
+interval: u64
}
Api --> ChitchatRef : uses
main --> Opt : parses CLI args
main --> Api : instantiates API
main --> ChitchatConfig : configures gossip node
main --> spawn_chitchat : spawns gossip node
main --> OpenApiService : creates HTTP API
main --> Server : runs HTTP server

This diagram represents the high-level structure showing the main structs (Api, Opt) and the main function interactions with configuration, gossip node spawning, API creation, and HTTP server execution. The Api struct depends on ChitchatRef for accessing gossip state.