Dockerfile.scratch.oc9

Overview

This Dockerfile defines a container image based on the opencloudos/opencloudos:9.0 base image, tailored to run the ragflow application and its associated services. It sets up a Python 3.11 environment using Miniconda, installs key system dependencies including OpenMPI, Node.js, and Nginx, and prepares the container for running a complex software stack that includes web, API, and machine learning components. The container is configured to execute a custom entrypoint script to launch the application.

Detailed Explanation

Base Image and User

FROM opencloudos/opencloudos:9.0
USER root

• Purpose: Uses OpenCloudOS version 9.0 as the base image, which is a RHEL-like Linux distribution optimized for cloud environments.

• USER root: All subsequent commands run as the root user for installation and setup.

Working Directory

WORKDIR /ragflow

• Sets the working directory inside the container to /ragflow, which is the root of the application source code.

System Package Installation

RUN dnf update -y && dnf install -y wget curl gcc-c++ openmpi-devel

• Updates all system packages.

• Installs essential development tools and OpenMPI development libraries required for distributed computing.

Miniconda Installation and Python Environment Setup

RUN wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh && \
    bash ~/miniconda.sh -b -p /root/miniconda3 && \
    rm ~/miniconda.sh && ln -s /root/miniconda3/etc/profile.d/conda.sh /etc/profile.d/conda.sh && \
    echo ". /root/miniconda3/etc/profile.d/conda.sh" >> ~/.bashrc && \
    echo "conda activate base" >> ~/.bashrc

ENV PATH /root/miniconda3/bin:$PATH

RUN conda create -y --name py11 python=3.11

ENV CONDA_DEFAULT_ENV py11
ENV CONDA_PREFIX /root/miniconda3/envs/py11
ENV PATH $CONDA_PREFIX/bin:$PATH

• Downloads and installs Miniconda silently to /root/miniconda3.

• Configures shell initialization scripts to enable conda commands.

• Adds Miniconda binaries to the PATH.

• Creates a new Conda environment named py11 with Python 3.11.

• Sets environment variables to use the py11 Conda environment by default inside the container.

Node.js and Nginx Installation

RUN dnf install -y nodejs
RUN dnf install -y nginx

• Installs Node.js for building frontend assets.

• Installs Nginx for serving web content or acting as a reverse proxy.

Adding Application Source Code

ADD ./web ./web
ADD ./api ./api
ADD ./docs ./docs
ADD ./conf ./conf
ADD ./deepdoc ./deepdoc
ADD ./rag ./rag
ADD ./requirements.txt ./requirements.txt
ADD ./agent ./agent
ADD ./graphrag ./graphrag
ADD ./plugin ./plugin

• Copies multiple directories and files from the build context into the container's working directory.

• These directories likely contain the frontend (web), backend API (api), documentation (docs), configuration files (conf), and various application modules.

Additional MPI and Python Packages Setup

RUN dnf install -y openmpi openmpi-devel python3-openmpi
ENV C_INCLUDE_PATH /usr/include/openmpi-x86_64:$C_INCLUDE_PATH
ENV LD_LIBRARY_PATH /usr/lib64/openmpi/lib:$LD_LIBRARY_PATH
RUN rm /root/miniconda3/envs/py11/compiler_compat/ld

• Installs OpenMPI runtime and Python bindings.

• Sets environment variables for MPI include and library paths to ensure correct compilation and linking.

• Removes a problematic ld compiler compatibility shim inside the Conda environment to avoid conflicts with system OpenMPI.

Frontend Build

RUN cd ./web && npm i && npm run build

• Changes directory to the frontend source.

• Installs Node.js dependencies with npm i.

• Builds the production frontend assets using npm run build.

Python Dependencies Installation

RUN conda run -n py11 pip install $(grep -ivE "mpi4py" ./requirements.txt) # without mpi4py==3.1.5
RUN conda run -n py11 pip install redis

• Installs Python packages from requirements.txt excluding the mpi4py package, which may have compatibility issues or is managed separately.

• Installs redis Python client explicitly, indicating Redis may be used for caching or message brokering.

Additional System Libraries and Python Packages

RUN dnf update -y && \
    dnf install -y glib2 mesa-libGL && \
    dnf clean all

RUN conda run -n py11 pip install ollama
RUN conda run -n py11 python -m nltk.downloader punkt
RUN conda run -n py11 python -m nltk.downloader wordnet

• Updates system packages and installs GLib2 and Mesa OpenGL libraries, likely required by some Python or native components.

• Installs ollama Python package (likely AI/ML related).

• Downloads NLTK corpora punkt and wordnet for natural language processing tasks.

Environment Variables for Application

ENV PYTHONPATH=/ragflow/
ENV HF_ENDPOINT=https://hf-mirror.com

• Sets PYTHONPATH to /ragflow/ to ensure application modules can be imported correctly.

• Sets an endpoint for Hugging Face model mirror, likely to speed up downloads or avoid rate limits.

Configuration and Entrypoint Script

COPY docker/service_conf.yaml.template ./conf/service_conf.yaml.template
ADD docker/entrypoint.sh ./entrypoint.sh
RUN chmod +x ./entrypoint.sh

ENTRYPOINT ["./entrypoint.sh"]

• Copies a service configuration template into the container.

• Adds an entrypoint shell script and makes it executable.

• Sets the entrypoint to run entrypoint.sh, which is expected to start the application or perform container initialization.

Important Implementation Details and Algorithms

• Selective Python Package Installation: The Dockerfile excludes mpi4py from the bulk pip install, possibly due to version conflicts or because MPI bindings are handled at the system package level, ensuring compatibility with system OpenMPI.

• OpenMPI Setup: Explicit environment variables and removal of Conda’s ld shim ensure that MPI-based components compile and link against the correct system libraries.

• Frontend Build Integration: The container builds the frontend assets inside the image, ensuring that the final container contains production-ready web assets without requiring build steps at runtime.

• Multi-language Stack: The image supports Python 3.11, Node.js, and system libraries, indicating a complex full-stack application.

Interaction with Other System Components

• Web and API Services: The copied directories web and api suggest the container runs both frontend and backend services.

• Nginx: Installed to serve static files or reverse proxy HTTP requests to backend services.

• Redis: Installed Python client implies Redis is used externally or within the system for caching or messaging.

• MPI-enabled Components: OpenMPI installation and environment setup indicate distributed or parallel computation capabilities.

• Hugging Face Models: Environment variable points to a Hugging Face mirror, suggesting the application downloads or serves ML models.

• Entrypoint Script: The entrypoint script likely initializes environment variables, starts services like Nginx, API server, or worker processes.

Usage Example

To build and run the container, assuming you have this Dockerfile named Dockerfile.scratch.oc9:

docker build -t ragflow-app -f Dockerfile.scratch.oc9 .
docker run -it --rm -p 80:80 ragflow-app

• This will build the image and run it, exposing port 80 for web access.

• The container will execute entrypoint.sh on start.

Visual Diagram of Workflow and Structure

flowchart TD
    A[Base Image: opencloudos:9.0] --> B[Update & Install System Packages]
    B --> C[Install Miniconda & Setup Python 3.11 Env]
    C --> D[Install Node.js & Nginx]
    D --> E[Copy Application Source Code]
    E --> F[Install OpenMPI & Set Env Variables]
    F --> G[Build Frontend (npm build)]
    G --> H[Install Python Dependencies (pip)]
    H --> I[Install Additional System Libraries & NLP Data]
    I --> J[Set Application Environment Variables]
    J --> K[Copy Config & Entrypoint Script]
    K --> L[Container Entrypoint: ./entrypoint.sh]

Summary

This Dockerfile sets up a complex environment to run the ragflow application, combining system-level components, a Python 3.11 Conda environment, Node.js frontend build steps, and MPI support. It carefully manages dependencies and environment variables to support high-performance distributed computing and web serving, encapsulated with an entrypoint script to start the application services.

The file is a crucial part of the deployment pipeline, enabling consistent, reproducible builds of the ragflow service stack optimized for OpenCloudOS 9.0 environments.