rr.go

Overview

The rr.go file implements the httprr package, which provides HTTP request recording and replaying capabilities primarily aimed at testing scenarios. It defines a mechanism to capture HTTP request-response pairs during a test run ("record" mode) and to serve those previously recorded responses later without making real network calls ("replay" mode). This approach allows deterministic, repeatable tests that depend on HTTP interactions.

The package's main type is RecordReplay, an implementation of the http.RoundTripper interface. Depending on a command-line flag (-httprecord), RecordReplay either records live HTTP traffic to a file or reads from a previously recorded file to replay responses. The recorded data is stored in a custom plain-text trace format.

Key Components and Their Functionality

Global Variables and Initialization

• record *string
  Pointer to a string flag determining which files to record traces for. It is set only during test runs (checked via testing.Testing()).

• init()
  Registers the -httprecord flag to specify trace recording files using regex filter.

RecordReplay Type

RecordReplay is the core type that implements HTTP recording and replaying:

type RecordReplay struct {
	file string            // Path to the trace file
	real http.RoundTripper // Actual RoundTripper used for real HTTP calls

	mu        sync.Mutex
	reqScrub  []func(*http.Request) error  // Request scrubbers to canonicalize requests
	respScrub []func(*bytes.Buffer) error  // Response scrubbers to canonicalize responses
	replay    map[string]string            // Replay map: request key → response log entry
	record    *os.File                     // File handle for recording mode
	writeErr  error                        // Error encountered during writing
}

• file: Name of the trace file used for recording or replaying.

• real: The underlying HTTP transport used for real network calls in record mode.

• reqScrub and respScrub: Lists of functions to sanitize or canonicalize requests and responses respectively before recording or replaying.

• replay: Map storing recorded requests and their associated responses for replay mode.

• record: File handle opened for writing when recording.

• writeErr: Any error encountered during writing to the trace file.

Methods on RecordReplay

ScrubReq

func (rr *RecordReplay) ScrubReq(scrubs ...func(req *http.Request) error)

Adds request scrubbing functions that preprocess requests before they are logged or used as replay keys. Useful for removing secrets or normalizing non-deterministic fields.

• Parameters: One or more functions of type func(*http.Request) error.

• Effect: Appends new scrubbers to existing ones; does not replace.

ScrubResp

func (rr *RecordReplay) ScrubResp(scrubs ...func(*bytes.Buffer) error)

Adds response scrubbing functions that preprocess the raw byte representation of HTTP responses before logging or replaying.

• Parameters: One or more functions of type func(*bytes.Buffer) error.

• Effect: Appends new scrubbers to existing ones.

Recording

func (rr *RecordReplay) Recording() bool

Returns true if the RecordReplay is in recording mode (i.e., writing to a trace file).

Open

func Open(file string, rt http.RoundTripper) (*RecordReplay, error)

Creates a new RecordReplay instance either in record or replay mode depending on the -httprecord flag and the file name.

• Parameters:

  • file: Trace file path.

  • rt: Underlying HTTP RoundTripper used for real HTTP requests during recording.

• Returns:
  Pointer to a RecordReplay instance or an error.

• Behavior:
  If -httprecord matches the file name, creates a new recording log file; otherwise, opens an existing log file for replay.

Recording (Package-level)

func Recording(file string) (bool, error)

Returns whether recording mode is enabled for the given file by matching -httprecord against the file name regex.

create (private)

func create(file string, rt http.RoundTripper) (*RecordReplay, error)

Creates a new file for recording, writes a header, and returns a recording RecordReplay.

open (private)

func open(file string, rt http.RoundTripper) (*RecordReplay, error)

Loads a previously recorded trace file and parses it into an in-memory map for replay.

• Reads the entire file.

• Validates the header line.

• Parses request-response pairs, storing them in replay map.

Client

func (rr *RecordReplay) Client() *http.Client

Returns an http.Client configured to use this RecordReplay instance as its Transport.

RoundTrip

func (rr *RecordReplay) RoundTrip(req *http.Request) (*http.Response, error)

Implements the http.RoundTripper interface.

• In replay mode: Looks up the request in the stored map and returns the cached response.

• In record mode: Sends the real request via the underlying RoundTripper, records the request and response to the log, and returns the real response.

The method uses reqWire and respWire helpers to serialize requests and responses into canonical wire formats suitable for logging and lookup.

reqWire (private)

func (rr *RecordReplay) reqWire(req *http.Request) (string, error)

Serializes a request into a canonical string key for logging or lookup.

• Clones the request.

• Reads and replaces the body with a custom *Body type.

• Applies all registered request scrubbers.

• Serializes the scrubbed request using WriteProxy.

respWire (private)

func (rr *RecordReplay) respWire(resp *http.Response) (string, error)

Serializes a response into a canonical byte-string for logging.

• Writes the response to a buffer.

• Reads it back to ensure a consistent copy.

• Applies all registered response scrubbers.

replayRoundTrip (private)

func (rr *RecordReplay) replayRoundTrip(req *http.Request, reqLog string) (*http.Response, error)

Given a serialized request key, looks up the cached response in replay and returns it as an *http.Response.

Returns an error if the request is not found or the response log is corrupt.

writeError (private)

func (rr *RecordReplay) writeError() error

Returns any existing error encountered during writing to the recording file.

writeLog (private)

func (rr *RecordReplay) writeLog(reqWire, respWire string) error

Appends a new request-response record to the recording log file.

• Writes the lengths of the request and response sections.

• Writes the serialized request and response.

• On error, closes the file, deletes the log, and stores the error to prevent further writes.

Close

func (rr *RecordReplay) Close() error

Closes the RecordReplay.

• In record mode: closes the log file.

• In replay mode: no-op.

Body Type

type Body struct {
	Data       []byte
	ReadOffset int
}

Custom implementation of io.ReadCloser for request bodies in scrubbers.

• Stores full request body bytes.

• Implements Read and Close methods to satisfy io.ReadCloser.

• Used to allow scrubbers to access and modify HTTP request bodies easily.

Implementation Details and Algorithms

• The trace files use a custom plain-text format starting with a line "httprr trace v1".

• Each record in the file consists of a line with two numbers n1 n2 indicating the byte lengths of the serialized request and response respectively, followed by the request bytes and then the response bytes.

• Requests and responses are serialized using the standard HTTP wire format (http.Request.WriteProxy for requests, http.Response.Write for responses).

• Scrubbing functions allow customization of the recorded data to remove secrets or normalize dynamic fields, ensuring deterministic matching and privacy.

• The replay mode stores all requests and responses in an in-memory map for efficient lookup.

• The record mode writes data incrementally to disk, handling write errors gracefully by aborting and deleting corrupted logs.

• The RoundTrip method decides mode dynamically based on whether the replay map or record file is set.

• The package uses a mutex to synchronize writes to the log file.

Interaction with Other Parts of the System

• The -httprecord flag controls whether recording mode is enabled, and is only defined during test runs (go test).

• RecordReplay implements the http.RoundTripper interface, so it can be used transparently in any HTTP client.

• The Client() method facilitates creating an http.Client using RecordReplay as its transport.

• The package reads and writes trace files that can be reused in subsequent test runs to simulate HTTP interactions without network calls.

• Scrubbers allow user code to customize and sanitize requests and responses before logging or replaying.

• The design supports testing environments that require deterministic HTTP interactions, such as unit tests or integration tests with external APIs.

Visual Diagram

classDiagram
class RecordReplay {
-file: string
-real: http.RoundTripper
-mu: sync.Mutex
-reqScrub: []func(*http.Request) error
-respScrub: []func(*bytes.Buffer) error
-replay: map[string]string
-record: *os.File
-writeErr: error
+ScrubReq(...)
+ScrubResp(...)
+Recording() bool
+Open(file, rt) *RecordReplay
+Client() *http.Client
+RoundTrip(req *http.Request) *http.Response
+Close()
}
class Body {
-Data: []byte
-ReadOffset: int
+Read(p []byte) (int, error)
+Close() error
}
RecordReplay --> http.RoundTripper : implements
RecordReplay o-- Body : uses for req.Body