feat: [Indexer] implemented pg store for indexer

[sadiq1971]

Closes #160

[gemini-code-assist]

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a comprehensive indexer for Canton ledger events, specifically tailored for CIP56 token transfers. It establishes a resilient streaming pipeline to consume ledger updates, processes and decodes token transfer events, and persistently stores the derived state in a PostgreSQL database. The indexed data is then made accessible through a dedicated service layer with a RESTful HTTP API, providing a robust foundation for querying token-related information.

Highlights

• New Canton Ledger Streaming Client: Introduced a new streaming package that provides a robust, generic client for consuming Canton ledger updates. This client handles automatic reconnection, exponential backoff on transient errors, and invalidation of authentication tokens on 401/403 responses, ensuring reliable data ingestion.
• Indexer Core Logic: Implemented the core components of the indexer within the engine package, including a decoder for CIP56 TokenTransferEvents, a fetcher to retrieve event batches from the streaming client, and a processor to manage the atomic persistence of these events and their derived state (tokens, balances, total supply, holder counts) into the database.
• PostgreSQL Persistence Layer: Developed a PostgreSQL-backed store (store/pg.go) using the Bun ORM, designed to handle the atomic storage of parsed events, token metadata, and party balances. This includes mechanisms for idempotency, transactional updates, and efficient querying.
• Indexer Service and HTTP API: Created a service layer that exposes the indexed data through a well-defined API, including HTTP endpoints for querying tokens, balances (per party and per token), and historical events. This service also incorporates logging for all API calls.
• Database Migrations: Added a new migrations/indexerdb package with schema migration scripts for setting up the necessary PostgreSQL tables (indexer_events, indexer_tokens, indexer_balances, indexer_offsets) and their respective indexes.
• DAML Value Decoding Utilities: Enhanced the pkg/cantonsdk/values/decode.go file with new utility functions to safely extract various DAML value types (Text, Party, Numeric, Timestamp, Record, Optional, TextMap) from lapiv2.Value protobufs, improving type safety and reducing direct protobuf dependency in higher-level code.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request introduces a well-structured and thoroughly tested PostgreSQL store for the indexer, which is a significant feature. The implementation using bun ORM, testcontainers for database testing, and clear separation into models, store logic, and migrations is excellent. I have two main points of feedback: one regarding a potential race condition in pagination logic that could lead to inconsistent data for clients, and another about enhancing the test setup to more closely mirror the production database schema by including index creation. Overall, this is a high-quality contribution.

[dhyaniarun1993]

Lint fix required here as well.