docker-compose-base.yml

Overview

docker-compose-base.yml is a Docker Compose configuration file designed to orchestrate a multi-service application stack primarily focused on data indexing, storage, and processing. It defines the deployment of core infrastructure services such as Elasticsearch, OpenSearch, MySQL, Redis, MinIO, and custom components like Infinity and Sandbox Executor Manager. Each service is configured with environment variables, resource limits, health checks, and network settings to ensure reliable operation within a Docker-based environment.

This base compose file is intended to provide foundational service definitions that can be extended or overridden by other compose files or profiles depending on deployment needs. It supports profile-based service activation, enabling selective startup of services based on the context (e.g., development, testing, or production).

Detailed Service Descriptions

1. Elasticsearch Service (es01)

• Purpose: Provides an Elasticsearch node for indexing and searching data.

• Container Name: ragflow-es-01

• Image: elasticsearch:${STACK_VERSION} (version controlled via environment variable)

• Profiles: elasticsearch

• Ports: Exposes Elasticsearch's default port 9200 mapped to ${ES_PORT} on the host.

• Volumes: Persists data in Docker volume esdata01.

• Environment Variables:

  • node.name=es01 — Node identifier within the cluster.

  • ELASTIC_PASSWORD — Password for the built-in elastic user.

  • Elasticsearch cluster and security settings, including disabling SSL for HTTP and transport layers.

  • Disk watermarks to control shard allocation.

  • TZ for timezone configuration.

• Resource Limits: Memory limit set via ${MEM_LIMIT} with unlimited memory lock (memlock).

• Healthcheck: Uses curl to check if Elasticsearch HTTP endpoint is responsive.

• Restart Policy: on-failure

• Network: Connected to the ragflow bridge network.

Usage example:

docker-compose -f docker-compose-base.yml --profile elasticsearch up -d

2. OpenSearch Service (opensearch01)

• Purpose: Runs an OpenSearch single-node cluster, an alternative to Elasticsearch.

• Container Name: ragflow-opensearch-01

• Image: hub.icert.top/opensearchproject/opensearch:2.19.1

• Profiles: opensearch

• Ports: Maps OpenSearch port 9201 to ${OS_PORT}.

• Volumes: Persisted data stored in osdata01.

• Environment Variables:

  • Node name, administrator password, security plugin configurations.

  • Disk allocation watermarks and timezone.

• Resource Limits: Memory limit and memlock settings similar to Elasticsearch service.

• Healthcheck: Checks HTTP endpoint on port 9201.

• Restart Policy: on-failure

• Network: Connected to ragflow.

3. Infinity Service (infinity)

• Purpose: Runs the Infinity service, typically a data processing or workflow engine.

• Container Name: ragflow-infinity

• Image: infiniflow/infinity:v0.6.0-dev5

• Profiles: infinity

• Ports: Exposes multiple ports including Thrift (23817), HTTP (23820), and PostgreSQL (5432), mapped to corresponding environment variables.

• Volumes:

  • Data persisted in infinity_data.

  • Configuration file infinity_conf.toml mounted from local directory.

• Command: Runs Infinity with the specified config file.

• Environment: Timezone configuration.

• Resource Limits: Memory limits and file descriptor limits (nofile set very high).

• Healthcheck: HTTP endpoint on port 23820 for node status.

• Restart Policy: on-failure

• Network: Connected to ragflow.

4. Sandbox Executor Manager (sandbox-executor-manager)

• Purpose: Manages sandboxed execution environments for running isolated jobs or tasks.

• Container Name: ragflow-sandbox-executor-manager

• Profiles: sandbox

• Image: Configurable via ${SANDBOX_EXECUTOR_MANAGER_IMAGE}, defaults to infiniflow/sandbox-executor-manager:latest.

• Ports: Exposes port 9385, configurable via ${SANDBOX_EXECUTOR_MANAGER_PORT}.

• Privileged: Runs with privileged access.

• Volumes: Mounts Docker socket for managing sibling containers.

• Security: no-new-privileges enabled to restrict escalation.

• Environment Variables: Controls pool size, base images for Python and NodeJS sandboxes, security options, memory, and timeout.

• Healthcheck: Checks HTTP health endpoint on port 9385.

• Restart Policy: on-failure

• Network: Connected to ragflow.

5. MySQL Service (mysql)

• Purpose: Provides a MySQL database for application data storage.

• Container Name: ragflow-mysql

• Image: mysql:8.0.39 (note: no official ARM64 5.7 image)

• Ports: Maps MySQL port 3306 to ${MYSQL_PORT}.

• Volumes: Persists data in mysql_data and mounts an initialization SQL script.

• Environment Variables: Sets root password and timezone.

• Command Options: Configures max connections, character set, collation, authentication plugin, TLS versions, and binlog expiration.

• Healthcheck: Uses mysqladmin ping with root credentials.

• Restart Policy: on-failure

• Network: Connected to ragflow.

6. MinIO Service (minio)

• Purpose: Object storage service compatible with Amazon S3 APIs.

• Container Name: ragflow-minio

• Image: quay.io/minio/minio:RELEASE.2025-06-13T11-33-47Z

• Ports: Exposes MinIO service on port 9000 and console UI on 9001.

• Volumes: Persists data in minio_data.

• Environment Variables: Root user and password, timezone.

• Command: Starts MinIO server with console UI on port 9001.

• Healthcheck: Checks MinIO liveness endpoint.

• Restart Policy: on-failure

• Network: Connected to ragflow.

7. Redis Service (redis)

• Purpose: Provides an in-memory key-value store for caching and fast data operations.

• Container Name: ragflow-redis

• Image: valkey/valkey:8 (a Redis 8 image variant)

• Ports: Maps Redis default port 6379 to ${REDIS_PORT}.

• Volumes: Persists data in redis_data.

• Command: Runs Redis server with password authentication, max memory of 128MB, and LRU eviction policy.

• Environment Variables: Password via .env, used in command.

• Healthcheck: Uses Redis CLI PING command with authentication.

• Restart Policy: on-failure

• Network: Connected to ragflow.

Volumes

The compose file defines several named volumes to persist data across container restarts:

• esdata01 - Elasticsearch data

• osdata01 - OpenSearch data

• infinity_data - Infinity service data

• mysql_data - MySQL database files

• minio_data - MinIO object storage

• redis_data - Redis data persistence

All volumes use the local driver.

Networks

• ragflow: A custom user-defined bridge network for inter-service communication. All services are connected to this network, enabling container name resolution and isolation from external Docker networks.

Important Implementation Details

• Profiles: Services are grouped under profiles (e.g., elasticsearch, opensearch, infinity, sandbox). This allows selective service startup by specifying profiles when running docker-compose.

• Healthchecks: Each service defines healthchecks with retries and intervals to ensure containers are only marked healthy when ready, aiding orchestration tools and load balancers.

• Environment Variables: Most configuration values are injected via environment variables and .env file, allowing flexible customization without modifying the compose file.

• Resource Limits: Memory limits and ulimits are set to prevent resource exhaustion and improve stability.

• Security: The sandbox executor manager runs privileged with restricted privileges (no-new-privileges) and mounts the Docker socket to manage containers dynamically.

• Data Persistence: Use of volumes ensures that container restarts or upgrades do not cause data loss.

• Single-node Discovery: Elasticsearch and OpenSearch are configured for single-node operation simplifying cluster setup.

Interactions with Other System Components

• This compose file forms the foundational infrastructure layer for applications requiring search (Elasticsearch/OpenSearch), relational database (MySQL), caching (Redis), object storage (MinIO), and data processing (Infinity).

• The Sandbox Executor Manager interacts with Docker to provision isolated task execution environments.

• Environment variables and .env provide integration points for external configuration management.

• Ports exposed by services enable external applications or other internal services to connect with these core components.

• The ragflow Docker network allows seamless inter-container communication.

Usage Example

To start the Elasticsearch and MySQL services only:

docker-compose -f docker-compose-base.yml --profile elasticsearch --profile mysql up -d

To bring up the full stack (assuming all profiles enabled):

docker-compose -f docker-compose-base.yml up -d

Mermaid Diagram

Below is a flowchart illustrating the relationships and key configurations between the services defined in this compose file.

flowchart TD
    subgraph "Network: ragflow"
        ES[Elasticsearch (es01)]
        OS[OpenSearch (opensearch01)]
        INF[Infinity]
        SANDBOX[Sandbox Executor Manager]
        MYSQL[MySQL]
        MINIO[MinIO]
        REDIS[Redis]
    end

    ES -- Data Volume --> esdata01[Volume: esdata01]
    OS -- Data Volume --> osdata01[Volume: osdata01]
    INF -- Data Volume --> infinity_data[Volume: infinity_data]
    MYSQL -- Data Volume --> mysql_data[Volume: mysql_data]
    MINIO -- Data Volume --> minio_data[Volume: minio_data]
    REDIS -- Data Volume --> redis_data[Volume: redis_data]

    SANDBOX -- Mounts --> docker_sock[/var/run/docker.sock]

    INF -->|Connects to| MYSQL
    INF -->|Uses| REDIS
    INF -->|Stores objects| MINIO
    SANDBOX -->|Manages| INF

    style esdata01 fill:#f9f,stroke:#333,stroke-width:1px
    style osdata01 fill:#f9f,stroke:#333,stroke-width:1px
    style infinity_data fill:#f9f,stroke:#333,stroke-width:1px
    style mysql_data fill:#f9f,stroke:#333,stroke-width:1px
    style minio_data fill:#f9f,stroke:#333,stroke-width:1px
    style redis_data fill:#f9f,stroke:#333,stroke-width:1px

Summary

docker-compose-base.yml is a robust, modular Docker Compose configuration that defines essential services for a data-centric application stack. It emphasizes configurability via environment variables, service health monitoring, resource management, and network isolation. The file is intended as a foundational layer to be extended or combined with other configurations for specific deployment scenarios.