Indexer v1.0.0 Release Notes

Highlights

We're excited to announce the dYdX Indexer’s v1.0.0 release, which provides support for the v1.0.0 dYdX Chain release. The indexer is a read-only service that serves on-chain and off-chain data to clients over REST APIs and Websockets. The indexer exists to index data emitted from a node and serve in a more performant, efficient, and user-friendly way that can be consumed by frontend applications.

On-chain vs Off-chain data

The indexer runs two separate ingestion flows for on-chain and off-chain data. The expected throughput of on-chain changes is 10-50 events/second, while the expected throughput of off-chain changes is 500-1,000 events/second. The separate flows exist to optimize for the different characteristics of these two datasets.

On-chain data is data that is stored on the blockchain and can be reproduced from past transactions. This includes:

• Account balances
• Account positions
• Order fills and associated rewards/fees
• Funding rate payments
• Historical oracle prices
• Order placement and cancellation (for stateful and conditional orders)
• Relevant governance events (e.g. liquidity tier changes)

Off-chain data is stored in-memory of each node. It is not committed to the blockchain. This includes:

• Order placement and cancellation (for short-term orders)
• Order book of each perpetual exchange pair
• Indexed order updates before they hit the chain

What's New?

In this inaugural release, we will briefly describe the purpose of each component of the indexer. In future releases, a detailed changelog will instead be included to summarize more granular additions or changes.

The indexer is made up of a series of services that ingest information from full nodes and serve that information to various clients. Kafka topics are used to pass events/data around to the services within the indexer. The key services that make up indexer are outlined below:

• Ender — Ingests on-chain data and stores it in Postgres. Emits events to be served by Socks.
• Vulcan — Ingests off-chain data and stores in a Redis cache. Emits events to be served by Socks.
• Comlink — Exposes REST APIs to serve indexer data.
• Socks — Exposes Websockets endpoint for streaming indexer data.
• Roundtable — Runs periodic jobs to aggregate exchange statistics.