Infinite Loading Support

Overview

The Infinite Loading Support module extends the core data fetching hook to enable fetching paginated or infinite lists of data in a declarative and efficient way. It solves the challenge of managing multiple pages of data that must be loaded progressively, concatenated, and cached, while maintaining synchronization with the global cache and minimizing unnecessary re-fetches.

By providing a specialized hook, useSWRInfinite, this module abstracts the complexity of managing page keys, page sizes, and revalidation logic across dynamic paginated data sets. It empowers developers to implement infinite scrolls, "load more" buttons, and similar UI patterns seamlessly integrated with SWR’s caching and revalidation mechanisms.

Core Concepts

1. Page-Based Data Fetching with Dynamic Keys

At the heart of infinite loading is the concept of fetching data page by page. Each page corresponds to a unique key generated by a key loader function, which takes the page index and optionally the previous page’s data to compute the key for the current page.

This key loader function enables conditional fetching (e.g., stop fetching when no more data) and supports complex pagination schemes (cursor-based, offset-based, etc.).

2. Centralized Page Size Management

The module manages the number of pages currently loaded via a page size state that can be read and updated through the size and setSize API. This state is cached globally to persist across component remounts and share across hooks with the same key.

3. Seamless Integration with Core SWR

The infinite loading hook leverages the core useSWR hook but composes it to handle multiple pages’ keys and data collectively. It coordinates revalidation across pages, supports parallel or sequential fetching strategies, and exposes a unified API for the array of loaded page data.

How It Works

`useSWRInfinite` Hook

This hook is implemented by composing the core useSWR hook with an infinite loading middleware (the infinite function). The middleware adds the infinite loading logic on top of the standard SWR behaviors.

const useSWRInfinite = withMiddleware(useSWR, infinite) as SWRInfiniteHook

Key Loader and Key Serialization

The hook receives a key loader function (getKey), which takes (pageIndex, previousPageData) and returns the key arguments for that page.
The first page’s key is serialized and prefixed with a special infinite prefix to create a unique infinite key used for caching page size and metadata.

const infiniteKey = INFINITE_PREFIX + getFirstPageKey(getKey)

This infinite key is the anchor for storing page size and control flags in the cache.

Page Size and Cache State

The current page size (size) is managed in the cache under the infinite key’s metadata.
The hook exposes size and setSize:
- size: The number of pages currently loaded.
- setSize: A function to update the page size, which triggers fetching or discarding pages accordingly.
The page size is persisted or reset based on configuration (persistSize flag).

Data Fetching Workflow

When the hook is mounted or revalidated:

It determines the number of pages to load (pageSize).
For each page index up to pageSize, it calls the key loader to get the page key and arguments.
For each valid page key:
- It attempts to read cached data for that page.
- It decides whether to fetch fresh data for that page based on:
  - Configuration flags (revalidateAll, revalidateFirstPage, revalidateOnMount).
  - Cache state and whether the page data changed.
  - A custom shouldRevalidatePage function if provided.
It fetches data pages either sequentially or in parallel depending on the parallel config flag.
It stores each page’s data individually in the cache.
Finally, it returns an array of page data.

Mutate and Revalidation

The hook exposes a mutate function extended from SWR’s mutate, which supports:
- Mutating the entire pages array.
- Triggering revalidation for all or selected pages.
Internally, the mutate function sets flags in cache metadata to control which pages should revalidate.

React State and Subscriptions

The module uses useSyncExternalStore to subscribe to changes in the cache metadata (page size) and trigger component updates accordingly.
It uses refs and layout effects to track mount state and reset page size appropriately when the key changes.

Interactions with Other System Parts

Core SWR Hook (useSWR): The infinite loading module wraps and composes the core data fetching hook, extending its API and behavior without modifying its internals.
Cache Provider: The infinite loading state, including page size and individual page data, is stored and managed in the global cache. This allows cross-component sharing and persistence.
Serialization Utilities: The module uses serialization helpers to convert key loader outputs into stable string keys, critical for cache indexing and revalidation.
Global State and Preload Cache: It interacts with global state for managing preloaded requests, enabling integration with preloading mechanisms.

Design Patterns and Unique Approaches

Middleware Composition: Infinite loading is implemented as a middleware that wraps the core useSWR hook, showcasing the extensibility of the system via layered middleware.
Cache-Backed Page Size State: Instead of local React state, the module stores the page size in the shared cache with a special key. This enables consistent state across components using the same infinite key.
Dynamic Key Generation with Previous Page Data: The key loader function signature allows the next page’s key to depend on the previous page’s data, supporting cursor-based pagination.
Parallel vs. Sequential Fetching: The module supports configurable parallel fetching of pages for performance optimization or sequential fetching when order matters.
Selective Revalidation Control: Through cache metadata flags and configuration, it supports fine-grained control over which pages revalidate during mutation or re-fetch.

Key Code Snippets

Infinite Key Creation with Serialization

let infiniteKey: string | undefined
try {
  infiniteKey = getFirstPageKey(getKey)
  if (infiniteKey) infiniteKey = INFINITE_PREFIX + infiniteKey
} catch (err) {
  // Not ready yet.
}

Page Size Subscription Using `useSyncExternalStore`

const getSnapshot = useCallback(() => {
  const size = isUndefined(get()._l) ? initialSize : get()._l
  return size
}, [cache, infiniteKey, initialSize])

useSyncExternalStore(
  useCallback(
    (callback: () => void) => {
      if (infiniteKey)
        return subscribeCache(infiniteKey, () => {
          callback()
        })
      return () => {}
    },
    [cache, infiniteKey]
  ),
  getSnapshot,
  getSnapshot
)

Fetching Multiple Pages Sequentially or in Parallel

for (let i = 0; i < pageSize; ++i) {
  const [pageKey, pageArg] = serialize(
    getKey(i, parallel ? null : previousPageData)
  )
  if (!pageKey) break

  const revalidate = async () => {
    if (!hasPreloadedRequest) {
      pageData = await fn(pageArg)
    } else {
      pageData = await PRELOAD[pageKey]
      delete PRELOAD[pageKey]
    }
    setSWRCache({ data: pageData, _k: pageArg })
    data[i] = pageData
  }

  if (parallel) {
    revalidators.push(revalidate)
  } else {
    await revalidate()
  }

  if (!parallel) previousPageData = pageData
}

if (parallel) await Promise.all(revalidators.map(r => r()))

`setSize` API to Change Number of Loaded Pages

const setSize = useCallback(
  (arg: number | ((size: number) => number)) => {
    if (!infiniteKey) return EMPTY_PROMISE
    let size = isFunction(arg) ? arg(resolvePageSize()) : arg
    if (typeof size !== 'number') return EMPTY_PROMISE

    changeSize({ _l: size })
    lastPageSizeRef.current = size

    // Construct new page data array from cache or trigger mutate
    // ...
    return mutate(data)
  },
  [infiniteKey, cache, mutate, resolvePageSize]
)

Mermaid Sequence Diagram: Infinite Loading Data Fetching Workflow

sequenceDiagram
  participant Comp as React Component
  participant Hook as useSWRInfinite Hook
  participant Cache as Global Cache
  participant Fetcher as Data Fetcher

  Comp->>Hook: Mount with getKey and config
  Hook->>Cache: Read infiniteKey and pageSize metadata
  loop For each page index < pageSize
    Hook->>Hook: Generate pageKey with getKey(index, prevPageData)
    Hook->>Cache: Get cached pageData for pageKey
    alt Should fetch page
      Hook->>Fetcher: Fetch page data for pageKey
      Fetcher-->>Hook: Return page data
      Hook->>Cache: Cache page data
    else Use cached pageData
    end
    Hook->>Hook: Update pageData array
  end
  Hook-->>Comp: Return array of pageData

Summary

The Infinite Loading Support module provides a powerful extension to the core data fetching hook, enabling efficient and flexible loading of paginated data sets. Its design leverages middleware composition, cache-backed state, dynamic key generation, and configurable fetching strategies to solve the complexities of infinite scrolling and pagination within the SWR ecosystem.