dataset_example.sh

Overview

dataset_example.sh is a shell script that provides example usage of a RESTful API for managing datasets within the InfiniFlow system. The script demonstrates how to perform basic CRUD (Create, Read, Update, Delete) operations on datasets by invoking the corresponding HTTP endpoints using curl commands.

This file is primarily intended as a reference or starting point for developers or users who need to interact programmatically with the InfiniFlow dataset management API. It showcases the required HTTP methods, request payload formats, and necessary headers (including authorization) for dataset operations.

Detailed Explanation of Operations

The script contains four main sections, each corresponding to a dataset operation:

1. Create a Dataset

curl --request POST \
     --url http://localhost:9380/api/v1/datasets \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer <token>' \
     --data '{
      "name": "test"
      }'

Purpose: Creates a new dataset with the specified name.
HTTP Method: POST
Endpoint: /api/v1/datasets
Headers:
- Content-Type: application/json — indicates JSON payload.
- Authorization: Bearer <token> — bearer token for authentication.
Payload Parameters:
- name (string): The name of the dataset to create.
Expected Response: A JSON object representing the newly created dataset, usually including a unique identifier.

Example Usage:

To create a dataset named "test", you would run the above command with a valid authorization token.

2. Update a Dataset

curl --request PUT \
     --url http://localhost:9380/api/v1/datasets/2e898768a0bc11efb46a0242ac120006 \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer <token>' \
     --data '
     {
          "name": "updated_dataset"
     }'

Purpose: Updates the properties of an existing dataset identified by its unique ID.
HTTP Method: PUT
Endpoint: /api/v1/datasets/{dataset_id}
Headers:
- Content-Type: application/json
- Authorization: Bearer <token>
Parameters:
- dataset_id (string): The unique identifier of the dataset to update (in URL path).
Payload Parameters:
- name (string): The new name for the dataset.
Expected Response: A JSON object showing the updated dataset details.

Example Usage:

Update the dataset with ID 2e898768a0bc11efb46a0242ac120006 to have the name "updated_dataset".

3. List Datasets

curl --request GET \
     --url http://127.0.0.1:9380/api/v1/datasets \
     --header 'Authorization: Bearer <token>'

Purpose: Retrieves a list of all datasets accessible to the authenticated user.
HTTP Method: GET
Endpoint: /api/v1/datasets
Headers:
- Authorization: Bearer <token>
Parameters: None
Expected Response: A JSON array of dataset objects, each containing dataset metadata such as ID and name.

Example Usage:

List all datasets available to the user authenticated by the provided token.

4. Delete Datasets

curl --request DELETE \
     --url http://localhost:9380/api/v1/datasets \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer <token>' \
     --data '{
     "ids": ["301298b8a0bc11efa0440242ac120006"]
     }'

Purpose: Deletes one or more datasets specified by their IDs.
HTTP Method: DELETE
Endpoint: /api/v1/datasets
Headers:
- Content-Type: application/json
- Authorization: Bearer <token>
Payload Parameters:
- ids (array of strings): List of dataset IDs to delete.
Expected Response: Typically a success status or confirmation of deletion.

Example Usage:

Delete the dataset with ID 301298b8a0bc11efa0440242ac120006.

Important Implementation Details

Authorization: Each request requires an Authorization header with a Bearer token to authenticate the user. The token in the examples is a placeholder and should be replaced with valid credentials.
Content-Type: For requests with a body (POST, PUT, DELETE), the Content-Type header is set to application/json.
Endpoints: The script uses local endpoints (localhost or 127.0.0.1) with port 9380. This indicates that the InfiniFlow API server is expected to be running locally on this port.
Dataset Identification: Dataset update and delete operations reference datasets by a unique identifier (UUID-like string).
Shell Script Usage: The echo -e commands before each curl call print informative messages to the console to clarify the operation being performed.

Interaction With Other System Components

This script interacts directly with the InfiniFlow Dataset Management API.
The API server (presumably part of the InfiniFlow backend) handles dataset creation, update, retrieval, and deletion.
The dataset entities managed here could be used downstream in other InfiniFlow components for data processing, flow management, or analytics.
Authentication is handled externally (token generation is outside the scope of this script).
The script can be used as a basis for integration tests, automation scripts, or as documentation for API consumers.

Usage Summary

Operation	HTTP Method	Endpoint	Payload	Description
Create Dataset	POST	`/api/v1/datasets`	`{ "name": "dataset_name" }`	Create a new dataset
Update Dataset	PUT	`/api/v1/datasets/{dataset_id}`	`{ "name": "new_name" }`	Rename or update dataset metadata
List Datasets	GET	`/api/v1/datasets`	None	Retrieve list of datasets
Delete Datasets	DELETE	`/api/v1/datasets`	`{ "ids": ["id1", "id2"] }`	Delete one or more datasets

Mermaid Flowchart Diagram

The following flowchart illustrates the sequence and relationship of dataset operations as demonstrated in this script:

flowchart TD
    A[Start] --> B[Create Dataset]
    B --> C[Update Dataset]
    C --> D[List Datasets]
    D --> E[Delete Datasets]
    E --> F[End]

    subgraph Dataset API
      B
      C
      D
      E
    end

Summary

dataset_example.sh serves as a concise and practical example of using the InfiniFlow dataset REST API. It guides users through creating, updating, listing, and deleting datasets with properly formatted HTTP requests. It is a valuable resource for developers integrating with the InfiniFlow platform or performing automated dataset management tasks.