Skip to content

Add MIP-11 Support to DAS Indexer #268

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

Add MIP-11 Support to DAS Indexer #268

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
be82120
add MIP-11 support to DAS indexer
pileks Sep 30, 2025
5420abb
atomically add/delete group relations, update collection grouping ind…
pileks Oct 8, 2025
0f721e3
fix up asset_grouping query for groups, address sync issue
pileks Oct 9, 2025
eb69912
fmt
pileks Oct 9, 2025
889014d
add transaction-level advisory lock for asset_grouping manipulation o…
pileks Oct 9, 2025
cb378a6
Update cleanup DELETE to handle same-slot and NULL rows
pileks Oct 10, 2025
bc38abd
attempt simpler approach to handling core group plugin in asset_group…
pileks Oct 11, 2025
299e9e7
address suggestions
pileks Oct 13, 2025
70a268f
filter out groupings with null value
pileks Oct 16, 2025
04fcfc4
don't filter out null-value groupings for non-collections
pileks Oct 16, 2025
7310f57
fix coderabbit's found issue
pileks Oct 16, 2025
a2a3e13
fmt
pileks Oct 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions Cargo.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ migration = { path = "migration" }
mime_guess = "2.0.4"
mpl-bubblegum = "2.0.0"
mpl-account-compression = "0.4.2"
mpl-core = {version = "0.10.0", features = ["serde"]}
mpl-core = {version = "=0.10.1-alpha.2", features = ["serde"]}
Copy link

@coderabbitai coderabbitai bot Sep 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🔴 Critical

🧩 Analysis chain

Verify alpha version stability and consider version constraint.

Using an exact-pinned alpha version (=0.10.1-alpha.2) in the dependency may cause issues:

  1. Alpha version risk: Alpha releases are typically unstable and may contain bugs or breaking changes.
  2. Exact version pinning: The = constraint prevents any updates, even patch fixes. Consider using ~ for patch-level flexibility once a stable version is available.

Verify the stability of this alpha version and check if a stable release is available:

🌐 Web query:

What is the latest stable version of mpl-core Rust crate?

💡 Result:

The latest stable mpl-core crate version is 0.10.1 (checked Sep 30, 2025). [1][2]

Sources:
[1] docs.rs — mpl-core 0.10.1.
[2] lib.rs — mpl-core (shows version 0.10.1 in Cargo features).

Update mpl-core to the stable 0.10.1 release and loosen the version constraint.

Replace the exact alpha pin (=0.10.1-alpha.2) with a caret requirement (^0.10.1) to use the latest stable release and receive future patch fixes.

🤖 Prompt for AI Agents 
In Cargo.toml at line 72, the mpl-core dependency is pinned to the exact alpha
release "=0.10.1-alpha.2"; change this to a caret requirement "^0.10.1" so the
project uses the stable 0.10.1 release and allows compatible patch upgrades
(update the version string accordingly in that dependency entry).

mpl-noop = "0.2.1"
mpl-token-metadata = "4.1.1"
nft_ingester = { path = "nft_ingester" }
Expand Down
1 change: 1 addition & 0 deletions integration_tests/.gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
.env
59 changes: 57 additions & 2 deletions program_transformers/src/mpl_core_program/v1_asset.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,7 @@ use {
entity::{ActiveValue, ColumnTrait, EntityTrait},
prelude::*,
query::{JsonValue, QueryFilter, QuerySelect, QueryTrait},
sea_query::query::OnConflict,
sea_query::Expr,
sea_query::{query::OnConflict, Expr},
ConnectionTrait, CursorTrait, DbBackend, FromQueryResult, TransactionTrait,
},
serde_json::{value::Value, Map},
Expand Down Expand Up @@ -367,6 +366,62 @@ pub async fn save_v1_asset<T: ConnectionTrait + TransactionTrait>(
// asset_grouping table
//-----------------------

let last_group_slot_updated = asset_grouping::Entity::find()
.filter(
asset_grouping::Column::AssetId
.eq(id_vec.clone())
.and(asset_grouping::Column::GroupKey.eq("group".to_string())),
)
.one(&txn)
.await
.map_err(|db_err| ProgramTransformerError::AssetIndexError(db_err.to_string()))?
.map(|model| model.slot_updated.unwrap_or(0));

let last_group_slot_updated = last_group_slot_updated.unwrap_or(0);

// Only perform updates if the last asset_grouping for group plugin slot_updated is less than the current slot for the asset update.
if last_group_slot_updated < slot_i {
Copy link

@coderabbitai coderabbitai bot Sep 30, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick | 🔵 Trivial

Document the different slot comparison strategies.

There's an intentional difference in slot comparison logic between Groups and Collection groupings:

  • Line 383 (Groups): Uses strict less-than (<) to gate the entire update block, preventing unnecessary deletes/inserts when slots are equal.
  • Line 452 (Collection): Uses greater-or-equal (>=) in the WHERE clause, allowing updates when slots are equal.

This difference is likely intentional - Groups plugin requires delete-then-insert (no ON CONFLICT), while Collection uses upsert semantics. Consider adding a comment explaining this design choice for future maintainers.

-    // Only perform updates if the last asset_grouping for group plugin slot_updated is less than the current slot for the asset update.
+    // Only perform updates if the last asset_grouping for group plugin slot_updated is less than the current slot for the asset update.
+    // Note: We use strict < comparison here (vs >= for collections) because Groups requires delete-then-insert,
+    // so we skip the entire operation when slots are equal to avoid unnecessary database operations.
     if last_group_slot_updated < slot_i {

Also applies to: 452-453

🤖 Prompt for AI Agents 
In program_transformers/src/mpl_core_program/v1_asset.rs around lines 382-383
and 452-453, the slot comparison behavior differs: Groups uses strict less-than
(`<`) to skip updates when slots are equal, while Collection uses
greater-or-equal (`>=`) to allow updates when slots are equal; add a concise
comment at each location explaining this deliberate design choice — that Groups
performs delete-then-insert (no ON CONFLICT) so we must avoid touching records
when slots are equal, whereas Collection uses upsert semantics so equal-slot
updates are allowed — include the intent and the relation to delete-vs-upsert
behavior so future maintainers understand why the comparisons differ.

// Clear existing groupings for asset under the "group" key.
let query = asset_grouping::Entity::delete_many()
.filter(
asset_grouping::Column::AssetId
.eq(id_vec.clone())
.and(asset_grouping::Column::GroupKey.eq("group".to_string()))
.and(asset_grouping::Column::SlotUpdated.lt(slot_i)),
)
.build(DbBackend::Postgres);

txn.execute(query)
.await
.map_err(|db_err| ProgramTransformerError::AssetIndexError(db_err.to_string()))?;
Copy link
Contributor

@danenbm danenbm Oct 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want to have the delete and insert for this part of the same txn so if there's a failure they are reverted together

Copy link
Author

@pileks pileks Oct 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated the indexer to only fire off a single query for any of the cases - asset without group/empty group just deletes old groupings, while asset with groups now performs deletion and insertion using CTEs.


// Insert new groupings for asset under the "group" key.
if let Some(groups_plugin) = asset.plugins.get(&PluginType::Groups) {
if let Plugin::Groups(groups) = &groups_plugin.data {
// Skip empty groups plugin.
if !groups.groups.is_empty() {
let query =
asset_grouping::Entity::insert_many(groups.groups.iter().map(|group| {
asset_grouping::ActiveModel {
asset_id: ActiveValue::Set(id_vec.clone()),
group_key: ActiveValue::Set("group".to_string()),
group_value: ActiveValue::Set(Some(group.to_string())),
verified: ActiveValue::Set(true),
group_info_seq: ActiveValue::Set(Some(0)),
slot_updated: ActiveValue::Set(Some(slot_i)),
..Default::default()
}
}))
.build(DbBackend::Postgres);

txn.execute(query).await.map_err(|db_err| {
ProgramTransformerError::AssetIndexError(db_err.to_string())
})?;
}
}
}
}

if let UpdateAuthority::Collection(address) = asset.update_authority {
let model = asset_grouping::ActiveModel {
asset_id: ActiveValue::Set(id_vec.clone()),
Expand Down