baseline-1000rps.json

Overview

`baseline-1000rps.json` is a JSON-formatted data file representing detailed performance metrics and configuration details from a high-load test scenario, specifically a baseline test running at approximately 1000 requests per second (RPS). The file captures aggregated statistics about HTTP requests, iterations, virtual users (VUs), data throughput, and various timing metrics collected during the test run.

This file is primarily used for **performance analysis**, **benchmarking**, and **monitoring** within performance testing and load testing frameworks (e.g., k6 or similar tools). It provides key insights into the latency, throughput, error rates, and other vital statistics that help developers and testers understand system behavior under sustained load.

Detailed Explanation of File Structure and Contents

The file is structured as a single JSON object with the following main sections:

1. root_group

• Purpose: Represents the root grouping of checks and subgroups in the test scenario.

• Properties:

  • name (string): Name of the group; empty here indicating the root.

  • path (string): Path identifier; empty here.

  • id (string): Unique identifier hash for the group.

  • groups (array): Sub-groups within this root group. Empty here.

  • checks (array): Assertions or checks performed during the test. Empty here.

2. options

• Purpose: Configuration options related to summary statistics and output formatting.

• Properties:

  • summaryTimeUnit (string): Unit used for summarizing time metrics; empty here (likely defaults to milliseconds).

  • noColor (boolean): Whether to disable color output; false means colors are enabled.

  • summaryTrendStats (array of strings): Specifies statistical metrics to compute for trends, including:

    • Average (avg), minimum (min), median (med), maximum (max)

    • Percentiles at 90th (p(90)), 95th (p(95)), 99th (p(99))

    • Count (count)

3. state

• Purpose: Runtime state indicators during the test execution.

• Properties:

  • isStdErrTTY (boolean): Whether standard error stream is a TTY (interactive terminal).

  • testRunDurationMs (number): Duration of the test run in milliseconds (~120 seconds).

  • isStdOutTTY (boolean): Whether standard output stream is a TTY.

4. metrics

A comprehensive dictionary of performance metrics collected during the test, each with its own structure and statistical values.

Metric Types

• trend: Continuous numerical values varying over time, e.g., response times.

• counter: Counts or rates of events, e.g., number of iterations or data sent.

• gauge: Instantaneous measurements, e.g., number of virtual users.

• rate: Ratios or percentages, e.g., error rates.

Each metric typically contains:

• values: A dictionary of statistics (count, avg, percentiles).

• type: Metric type as above.

• contains: The kind of data this metric holds (e.g., "time", "default", "data").

• Optional thresholds: Performance thresholds and their pass/fail status.

Key Metrics Explained

• Iteration and Request Counts:

  • iterations (counter): Total number of test iterations and rate per second.

  • dropped_iterations (counter): Number and rate of iterations dropped.

  • http_reqs (counter): Total HTTP requests made and rate.

• Virtual Users (VUs):

  • vus (gauge): Current number of virtual users during test (~1000).

  • vus_max (gauge): Maximum number of virtual users reached (up to 3500).

• Timing Metrics (all in milliseconds):

  • iteration_duration (trend): Duration of each test iteration.

  • http_req_duration (trend): Total HTTP request duration.

  • http_req_tls_handshaking, http_req_connecting, http_req_blocked, http_req_sending, http_req_waiting, http_req_receiving (trend): Breakdown of HTTP request phases.

• Scenario-Specific Metrics:

  • http_req_duration{scenario:account} and http_req_duration{scenario:utxos}: Request durations broken down by scenario.

  • Corresponding error rates: http_req_failed{scenario:account} and http_req_failed{scenario:utxos}.

• Data Throughput:

  • data_received and data_sent (counter): Total bytes received and sent, and their rates.

• Error Rates:

  • http_req_failed (rate): Overall failure rate, passes, and fails count.

Usage Example

This JSON file is typically generated automatically by a load testing tool after running a test scenario. It can be consumed by:

• Performance dashboards to visualize test results.

• Automated scripts to assert thresholds and trigger alerts.

• Developers and testers analyzing system behavior post-test.

Example: Accessing Average HTTP Request Duration

const metrics = baselineData.metrics;
const avgHttpReqDuration = metrics["http_req_duration"].values.avg;
console.log(`Average HTTP request duration: ${avgHttpReqDuration} ms`);

Important Implementation Details and Algorithms

• Percentile Calculation:
  Percentiles such as p(90), p(95), and p(99) are used to understand latency distribution, indicating the value below which a given percentage of observations fall.

• Thresholds:
  The presence of thresholds (e.g., "p(95) < 800") indicates performance acceptance criteria. Metrics include pass/fail conditions to automatically flag if performance goals are met.

• Metric Types and Aggregation:
  The metrics are categorized to facilitate different aggregation strategies:

  • Trends: Calculated over time with statistical summaries.

  • Counters: Incremental counts or rates.

  • Gauges: Instantaneous values.

  • Rates: Ratios of events (e.g., error rates).

• Scenario Filtering:
  Metrics are sometimes scoped to particular test scenarios (e.g., account, utxos) allowing fine-grained performance analysis per user journey or API endpoint.

Interaction with Other System Components

• Load Testing Framework:
  This file serves as an output artifact from a load testing tool (like k6). It is generated post-test and consumed by reporting or alerting modules.

• Performance Dashboards and CI/CD Pipelines:
  The JSON data can be imported into monitoring systems or pipelines to track performance regressions or improvements over time.

• Test Orchestration:
  The options and state sections inform how the test was run and can be used to reproduce or debug test conditions.

Visual Diagram

The following flowchart illustrates the main metric categories and their relationships, highlighting how the file organizes and aggregates performance data.

flowchart LR
    A[baseline-1000rps.json] --> B[root_group]
    A --> C[options]
    A --> D[state]
    A --> E[metrics]

    E --> F[Trend Metrics]
    E --> G[Counter Metrics]
    E --> H[Gauge Metrics]
    E --> I[Rate Metrics]

    F --> F1[http_req_duration]
    F --> F2[iteration_duration]
    F --> F3[http_req_tls_handshaking]
    F --> F4[http_req_connecting]
    F --> F5[http_req_blocked]
    F --> F6[http_req_sending]
    F --> F7[http_req_waiting]
    F --> F8[http_req_receiving]
    F --> F9[scenario-specific durations]

    G --> G1[iterations]
    G --> G2[dropped_iterations]
    G --> G3[http_reqs]
    G --> G4[data_received]
    G --> G5[data_sent]

    H --> H1[vus]
    H --> H2[vus_max]

    I --> I1[http_req_failed]
    I --> I2[scenario-specific failures]

Summary

• baseline-1000rps.json is a detailed performance metrics report from a load test simulating ~1000 RPS.

• It contains structured data on request timings, counts, error rates, virtual users, and data throughput.

• The file supports performance validation through thresholds and scenario-specific breakdowns.

• It integrates with test frameworks and monitoring tools for performance analysis and continuous improvement.

This file is foundational for understanding system behavior under load and optimizing for scalability and reliability.