ipfs.ts

Overview

The `ipfs.ts` file is a Kubernetes infrastructure deployment script implemented using Pulumi and TypeScript. Its primary purpose is to automate the deployment of an IPFS (InterPlanetary File System) cluster within a Kubernetes namespace. This deployment provides a decentralized, fault-tolerant, and scalable file storage solution integrated into a Kubernetes environment.

The script handles the provisioning of all necessary Kubernetes resources, including Secrets, ConfigMaps, StatefulSets, Services, TLS certificates, and ingress routing to expose the IPFS gateway securely over HTTP(S). It configures both the IPFS daemon and the IPFS Cluster service, setting up storage volumes, network ports, health checks, and environment variables to ensure a robust IPFS cluster suitable for production usage.

This file is designed to be used as part of a broader infrastructure automation system, typically invoked during cluster setup or application deployment workflows that require decentralized file storage capabilities.

Detailed Documentation

Interface: IpfsClusterArgs

Defines the expected arguments for the IPFS deployment function.

Function: deployIpfs

function deployIpfs({ namespace, provider, domain, additionalDomain }: IpfsClusterArgs): void

Deploys a fully featured IPFS cluster within a specified Kubernetes namespace using Pulumi.

Parameters

• namespace (string): The Kubernetes namespace where resources will be deployed.

• provider (k8s.Provider): Pulumi Kubernetes provider for resource creation.

• domain (string): Primary domain for cert issuance and ingress.

• additionalDomain? (string): Optional secondary domain for ingress rules.

Returns

• void: This function creates Kubernetes resources but does not return a value.

Usage Example

import * as k8s from '@pulumi/kubernetes'

const provider = new k8s.Provider('k8s-provider', { /* ...connection config... */ })

deployIpfs({
  namespace: 'unchained-infra',
  provider,
  domain: 'example.com',
  additionalDomain: 'example.org',
})

Implementation Details

The function `deployIpfs` orchestrates the following Kubernetes resource creations and configurations:

1. Secret Creation

   • Stores the IPFS cluster secret, pulled from environment variables (IPFS_CLUSTER_SECRET).

   • Used for securing cluster communication.

2. ConfigMap Creation

   • Contains:

     • A fixed bootstrap peer ID for cluster initialization.

     • Two shell scripts (entrypoint.sh and configure-ipfs.sh) read from the local filesystem; these scripts configure the IPFS environment inside the pods.

3. Pod Specification

   • Defines a pod template with:

     • An init container (configure-ipfs) that runs the configuration script before the main containers start.

     • Two containers:

       • ipfs: Runs the IPFS daemon with appropriate ports exposed for swarm, API, HTTP gateway, and WebSocket communications.

       • ipfs-cluster: Runs the IPFS cluster management daemon coordinating pinning and replication.

     • Liveness probes on both containers to ensure health.

     • Volume mounts for persistent storage and configuration scripts.

     • Resource limits to control CPU and memory usage.

4. Service Creation

   • A ClusterIP service exposing all relevant IPFS and cluster ports to enable intra-cluster communication and proxying.

5. StatefulSet Creation

   • Deploys a 3-replica StatefulSet to manage the lifecycle of the IPFS cluster pods.

   • Uses Parallel pod management policy and RollingUpdate strategy for updates.

   • Defines PersistentVolumeClaims for:

     • cluster-storage (5Gi gp3 volume) for IPFS cluster state.

     • ipfs-storage (200Gi gp3 volume) for IPFS data.

6. Certificate Creation (cert-manager)

   • Creates a Certificate resource for TLS termination using Let's Encrypt ClusterIssuer.

   • Configured for the IPFS gateway domain.

7. IngressRoute Creation (Traefik)

   • Defines Traefik ingress routing rules for HTTP/HTTPS entry points.

   • Supports multiple domains if additionalDomain is provided.

   • Routes traffic to the IPFS HTTP gateway service port (8080).

   • Uses TLS with the created certificate.

8. Legacy Ingress Resource (Kubernetes)

   • Creates a simple Kubernetes Ingress resource for the gateway domain.

   • This is likely for compatibility or fallback purposes.

Important Implementation Notes

• Secrets Management: The cluster secret is injected from the environment variable IPFS_CLUSTER_SECRET. This secret must be securely managed outside of this script.

• Scripts Injection: The shell scripts for configuring IPFS are embedded into the ConfigMap and mounted as executable volumes within pods, allowing dynamic container configuration at startup.

• Pod Security Context: Both containers run as root (runAsUser: 0), which may be required due to the IPFS container images or volume permissions.

• Health Monitoring: Liveness probes use TCP socket checks on specific ports to detect if the containers are alive and restart if necessary.

• Storage Class: Persistent volumes use the gp3 storage class, which is typically associated with AWS EBS gp3 volumes. This may need adjustment for other environments.

• Rolling Updates: StatefulSet update strategy is rolling, minimizing downtime during upgrades.

• Ingress Routing: Traefik's IngressRoute CRD is used for advanced routing and TLS termination, complemented by a standard Kubernetes Ingress resource.

Interaction with Other System Components

• Pulumi Kubernetes Provider: This file depends on an external Pulumi Kubernetes provider instance to create resources within the specified cluster and namespace.

• Cluster Setup Automation: It is intended to be invoked as part of a larger deployment automation process that sets up Kubernetes clusters and namespaces.

• Certificate Management: Relies on cert-manager and a ClusterIssuer named lets-encrypt already configured in the cluster to issue TLS certificates.

• Traefik Ingress Controller: Requires Traefik to be deployed in the cluster to interpret the IngressRoute resources and route traffic accordingly.

• Environment Variables: The IPFS cluster secret is expected to be passed via environment configuration to ensure security.

• Filesystem Scripts: The deployment reads shell scripts (entrypoint.sh and configure-ipfs.sh) from the local file system relative to this file, ensuring IPFS containers are correctly configured at runtime.

Mermaid Diagram

Class/Function Structure Diagram

flowchart TD
    A[deployIpfs(args: IpfsClusterArgs)] --> B[Create Secret (cluster-secret)]
    A --> C[Create ConfigMap (scripts + bootstrap-peer-id)]
    A --> D[Define podSpec (initContainers + containers + volumes)]
    D --> D1[Init Container: configure-ipfs]
    D --> D2[Container: ipfs daemon]
    D --> D3[Container: ipfs-cluster service]
    A --> E[Create Service (ClusterIP, exposes IPFS ports)]
    A --> F[Create StatefulSet (3 replicas, volume claims)]
    A --> G[Create Certificate (cert-manager)]
    A --> H[Create IngressRoute (Traefik)]
    A --> I[Create Kubernetes Ingress]

Summary

The `ipfs.ts` file implements a comprehensive deployment of an IPFS cluster on Kubernetes using Pulumi. It sets up all necessary Kubernetes resources—secrets, configmaps, services, StatefulSets, and ingress rules—along with persistent storage and TLS certificates to provide a secure and scalable decentralized file storage system. This deployment is designed to integrate tightly with cluster provisioning and infrastructure automation workflows, enabling blockchain and distributed applications to leverage IPFS storage seamlessly and securely within a Kubernetes environment.

Appendix: File Layout Snapshot

If you require additional details on specific scripts (`entrypoint.sh` or `configure-ipfs.sh`), or integration examples, please let me know!