baseline-500rps.json

Overview

The `baseline-500rps.json` file is a structured JSON configuration and results snapshot primarily used in load testing or performance testing scenarios. It appears to capture detailed metrics and state information from a test run targeting approximately 500 requests per second (RPS). The file aggregates runtime statistics, performance metrics, error rates, and configuration options that facilitate the analysis of system behavior under load.

This file serves as an output artifact for performance testing tools (e.g., k6 or similar), providing comprehensive time-series and statistical data about HTTP requests and virtual user activity during the test execution. It is useful for performance engineers and developers to identify bottlenecks, verify system stability, and ensure that service-level objectives (SLOs) are met.

Detailed Explanation of File Structure and Contents

The file contains four primary top-level sections:

• root_group

• options

• state

• metrics

Each section is explained below.

1. root_group

{
  "id": "d41d8cd98f00b204e9800998ecf8427e",
  "groups": [],
  "checks": [],
  "name": "",
  "path": ""
}

• Purpose:
  Represents the root grouping of test scenarios or requests. It is an organizational container for grouping multiple performance checks or sub-groups of requests.

• Fields:

  • id (string): Unique identifier for the group (here appears as an MD5 hash of an empty string).

  • groups (array): Subgroups of requests or scenarios; empty here.

  • checks (array): Performance checks/assertions related to this group; empty here.

  • name (string): Name of the group; empty in this file.

  • path (string): Path or hierarchical location; empty here.

• Usage Example:
  In a more complex test, root_group would include multiple named groups representing different test scenarios or endpoints.

2. options

{
  "summaryTrendStats": [
    "avg", "min", "med", "max", "p(90)", "p(95)", "p(99)", "count"
  ],
  "summaryTimeUnit": "",
  "noColor": false
}

• Purpose:
  Configuration options dictating how test summary statistics are displayed or computed.

• Fields:

  • summaryTrendStats (array of strings): Specifies which statistical metrics to include in summary trend reports.
    Metrics include average (avg), minimum (min), median (med), maximum (max), and percentile values (p(90), p(95), p(99)), and count.

  • summaryTimeUnit (string): Unit for time-based metrics in summaries; empty implies default or milliseconds.

  • noColor (boolean): If true, disables colored output formatting in CLI or reports.

• Usage:
  Used by reporting tools to determine which statistics to calculate and how to format output.

3. state

{
  "isStdErrTTY": true,
  "testRunDurationMs": 120143.243455,
  "isStdOutTTY": true
}

• Purpose:
  Captures the runtime environment and metadata about the test execution.

• Fields:

  • isStdErrTTY (boolean): Indicates if the standard error output is a terminal (TTY).

  • testRunDurationMs (float): Total duration of the test run in milliseconds (~120 seconds here).

  • isStdOutTTY (boolean): Indicates if the standard output is a terminal.

• Usage:
  Useful for tooling to adapt output formatting and to report the test duration.

4. metrics

• Purpose:
  Contains detailed performance metrics collected during the test run, grouped by metric name and optionally tagged with scenario or other labels.

• Structure:
  Each metric is an object with:

  • type: Type of metric (e.g., "rate", "trend", "counter", "gauge").

  • contains: What the metric represents (e.g., "time", "default", "data").

  • values: Statistical values for the metric.

  • (Optional) thresholds: Pass/fail criteria for the metric.

• Common Metrics:

  • http_req_failed (rate): Rate of failed HTTP requests.

    • Values include rate, passes, fails.

  • http_req_duration (trend): Timing of HTTP requests with percentile breakdowns.

    • Values include min, med, max, avg, p(90), p(95), p(99), count.

  • http_req_blocked, http_req_connecting, http_req_sending, http_req_receiving, http_req_waiting, http_req_tls_handshaking (trend): Various sub-phases of HTTP request lifecycle timing.

  • iterations (counter): Number of test iterations executed and iteration rate.

  • dropped_iterations (counter): Number and rate of iterations dropped/skipped.

  • data_sent, data_received (counter): Volume of data transferred during the test.

  • vus, vus_max (gauge): Number of Virtual Users (VUs) active and maximum.

  • Scenario-specific Metrics: Metrics tagged with scenario labels like scenario:utxos and scenario:account to provide scoped performance data.

• Thresholds:
  Some metrics include thresholds defining pass/fail conditions (e.g., "p(95) < 800" ms for acceptable latency).

Important Implementation Details and Algorithms

• Metric Types:

  • Rate: Measures success/failure rates or event frequencies.

  • Trend: Measures continuous value changes over time, typically durations, with statistical percentiles.

  • Counter: Aggregates counts or totals over the test duration.

  • Gauge: Represents instantaneous values or ranges (e.g., number of active VUs).

• Statistical Metrics:

  • Percentiles (p(90), p(95), p(99)) represent distribution cutoffs, essential for assessing latency spikes.

  • Median (med) and averages (avg) help understand central tendency.

  • Min/max capture extremes.

• Thresholds:

  • Embedded pass/fail thresholds allow automated result evaluation without manual inspection.

• Scenario Tagging:

  • Metrics can be scoped to individual test scenarios, enabling fine-grained analysis of different workloads within the same test.

Interaction with Other System Components

• Load Testing Tool:
  This JSON file is most likely produced by a load testing execution engine (e.g., k6), which consumes a test script defining scenarios and HTTP requests.

• Reporting & Analysis:
  After test execution, this file is consumed by performance dashboards, CI/CD pipelines, or manual analysis tools to:

  • Visualize performance trends.

  • Detect anomalies or regression.

  • Generate reports for stakeholders.

• Continuous Integration (CI):
  Thresholds in metrics may be used to automatically gate deployments by failing builds if performance targets are not met.

• Test Scripts:
  The metrics correspond to requests and scenarios defined in test scripts that simulate user behavior against a target system.

Example Usage

Suppose a performance engineer wants to verify that the 95th percentile HTTP request duration for the "account" scenario is below 800 ms:

"http_req_duration{scenario:account}": {
  "thresholds": {
    "p(95) < 800": {
      "ok": true
    }
  }
}

This indicates the test passed the latency target, confirming acceptable performance.

Visual Diagram

The following **Mermaid flowchart** depicts the relationship between the main sections and key metric types within the `baseline-500rps.json` file, illustrating the flow of data from configuration, through state, to metrics collection.

flowchart TD
    A[baseline-500rps.json] --> B[root_group]
    A --> C[options]
    A --> D[state]
    A --> E[metrics]

    E --> F[rate Metrics]
    E --> G[trend Metrics]
    E --> H[counter Metrics]
    E --> I[gauge Metrics]

    F --> F1[http_req_failed]
    F --> F2[http_req_failed{scenario}]
    F --> F3[iterations]
    F --> F4[dropped_iterations]

    G --> G1[http_req_duration]
    G --> G2[http_req_blocked]
    G --> G3[http_req_connecting]
    G --> G4[http_req_sending]
    G --> G5[http_req_receiving]
    G --> G6[http_req_waiting]
    G --> G7[http_req_tls_handshaking]
    G --> G8[iteration_duration]

    H --> H1[data_sent]
    H --> H2[data_received]
    H --> H3[http_reqs]

    I --> I1[vus]
    I --> I2[vus_max]

    style A fill:#f9f,stroke:#333,stroke-width:2px
    style E fill:#bbf,stroke:#333,stroke-width:1px

Summary

The `baseline-500rps.json` file is a comprehensive data snapshot capturing the performance test results of a system under a load of approximately 500 requests per second. It contains configuration options, runtime state, and a rich set of metrics describing request success/failure rates, timing distributions, throughput, and resource utilization. The file is integral to performance testing workflows for analysis, reporting, and automated validation of service-level objectives.

*End of Documentation*