Affiliate Fee Indexer

Purpose

The Affiliate Fee Indexer addresses the need to accurately track and aggregate outbound affiliate fee transactions within the Thorchain blockchain ecosystem. Thorchain distributes affiliate fees to specific affiliate addresses whenever transactions pass through the network. This subtopic is dedicated to indexing these outbound affiliate fee transactions by scanning blockchain events and maintaining a comprehensive ledger of all affiliate revenue. This enables transparent reporting and auditing of affiliate fees, which is critical for stakeholders relying on affiliate revenue data.

Functionality

At its core, the Affiliate Fee Indexer continuously monitors new blocks and searches historic blocks for outbound transactions sent to a predefined list of affiliate addresses. Its main workflows include:

Initialization and Subscription:
- Upon creation, the indexer registers a block event handler via a WebSocket client that triggers processing for every new block.
Historic Synchronization (Sync method):
- Performs a paginated search over past blocks using multiple HTTP clients.
- Queries all transactions with outbound transfers directed to affiliate addresses.
- Utilizes concurrency with worker pools to efficiently fetch and process blocks and their results.
Block Processing:
- For each block, parses blockchain events to identify outbound transactions to affiliate addresses.
- Extracts relevant details such as amount, asset type, transaction ID, block height, block hash, timestamp, and recipient address.
- Stores each found affiliate fee record in an internal thread-safe collection.
Real-time Indexing:
- Listens for new blocks via WebSocket and immediately processes affiliate fee events as they appear.

This design ensures both historic and live affiliate fee transactions are indexed accurately and efficiently.

Key Data Flow

Block Search: Query blockchain nodes for transactions outbound to affiliate addresses.
Block Results Fetch: Retrieve detailed block results including events.
Event Parsing: Identify outbound affiliate fee events.
Record Creation: Construct AffiliateFee records with transaction and block metadata.
Record Storage: Append records safely to the indexer's fee list.

Critical Snippet Illustrating Event Handling

func (i *AffiliateFeeIndexer) processAffiliateFees(block thorchain.Block, blockEvents []cosmos.ABCIEvent, affiliateAddresses []string) {
  // Parse events from block
  _, typedEvents, err := thorchain.ParseBlockEvents(blockEvents)
  if err != nil {
    logger.Panicf("failed to parse block events for block: %d: %+v", block.Height(), err)
  }

  // Iterate over events to find outbound affiliate fees
  for _, event := range typedEvents {
    switch v := event.(type) {
    case *thorchain.EventOutbound:
      // Check if recipient is an affiliate address
      for _, affiliateAddress := range affiliateAddresses {
        if v.To == affiliateAddress {
          // Create and store affiliate fee record
          affiliateFee := &AffiliateFee{
            TxID:        v.InTxID,
            Address:     v.To,
            Amount:      strings.Fields(v.Coin)[0],
            Asset:       strings.Fields(v.Coin)[1],
            BlockHeight: block.Height(),
            BlockHash:   block.Hash(),
            Timestamp:   block.Timestamp(),
          }
          i.mu.Lock()
          i.AffiliateFees = append(i.AffiliateFees, affiliateFee)
          i.mu.Unlock()
          break
        }
      }
    }
  }
}

Relationship to Parent Topic and Other Subtopics

The Affiliate Fee Indexer is a specialized component within the broader Thorchain-specific functionality, complementing other subtopics such as:

Transaction Parsing: While general transaction parsing interprets diverse Thorchain and Cosmos SDK messages and events, the Affiliate Fee Indexer focuses exclusively on outbound affiliate fee events and their aggregation.
API Server: The API server exposes endpoints that provide access to Thorchain blockchain data, including affiliate revenue. It relies on the Affiliate Fee Indexer’s data as the authoritative source for affiliate fee transactions.

Together, these components form a cohesive system for processing, indexing, and exposing detailed Thorchain blockchain data with a focus on affiliate revenue transparency.

Unlike the generic transaction parsing subtopic, this indexer introduces concurrency and pagination in historic block searching and maintains an internal store of affiliate fees, providing optimized, real-time, and historical affiliate fee tracking not available in other subtopics.

Diagram: Affiliate Fee Indexer Workflow

flowchart TD
    Start[Start Indexer / On New Block] --> CheckNewBlock{New Block Received?}
    CheckNewBlock -- Yes --> FetchBlockEvents[Fetch Block Events]
    FetchBlockEvents --> ParseEvents[Parse Block Events]
    ParseEvents --> IdentifyAffiliateFees{Outbound To Affiliate?}
    IdentifyAffiliateFees -- Yes --> CreateFeeRecord[Create Affiliate Fee Record]
    CreateFeeRecord --> StoreRecord[Store Record Thread-Safely]
    IdentifyAffiliateFees -- No --> EndProcess[End Processing]
    StoreRecord --> EndProcess

    StartHistoricSync[Start Historic Sync] --> ForEachAffiliate[For Each Affiliate Address]
    ForEachAffiliate --> PaginatedBlockSearch[Paginate Blocks with Outbound Tx to Affiliate]
    PaginatedBlockSearch --> FetchBlockResults[Fetch Block Results]
    FetchBlockResults --> ParseEventsHistoric[Parse Events in Blocks]
    ParseEventsHistoric --> IdentifyAffiliateFeesHistoric{Outbound To Affiliate?}
    IdentifyAffiliateFeesHistoric -- Yes --> CreateFeeRecordHistoric[Create Fee Record]
    CreateFeeRecordHistoric --> StoreRecord
    IdentifyAffiliateFeesHistoric -- No --> Continue
    Continue --> EndHistoricSync[End Historic Sync]

    EndProcess -.-> WaitForNextBlock[Wait for Next Block]
    WaitForNextBlock --> CheckNewBlock

This flowchart illustrates both the live block processing triggered by new blocks and the historic synchronization process that paginates through past blocks to index all affiliate fee transactions.

The Affiliate Fee Indexer ensures Thorchain’s affiliate revenue data is kept current and comprehensive, enabling transparent tracking and supporting analytics and reporting services built on top of this indexed data.