-
Notifications
You must be signed in to change notification settings - Fork 49
Description
Currently the DAS API stack relies on having a geyser plugin indexer. One proposal would be to add/transition to a geyser grpc based indexer (https://github.com/rpcpool/yellowstone-grpc). This would allow:
- Multiple parallel indexers connected to different grpc endpoints (even different providers)
- Better failover
- Lower cost since Yellowstone gRPC can be delivered as a shared service.
For account updates and transaction updates this should be pretty straight forward, you can subscribe to confirmed or finalized commitment level via gRPC, convert to Plerkle format and push to redis.
You could also implement a block check mode which checks that you have indexed all transactions from a given block and if NOT, fetch that block via gRPC to fill in missing TXs.
There is experimental support in yellowstone-grpc for fetching program account state which would allow snapshots of the Tokenkeg account to be taken for indexing of traditional NFTs. Snapshots could also be handled through one time ingest through a tool like https://github.com/terorie/solana-snapshot-etl which could even run the plerkle plugin to update all accounts at once. With snapshot-etl the order would be:
- Start up the geyser grpc ingestion
- Run snapshot-etl to fill in all the missing assets while grpc subscriptions are running.