Named Storage Slots

[PhilippGackstatter]

I'm wondering if the public interface of an account should be that storage slots are named instead of indexed. For example, use a felt-encoded representation of the string "miden.basic_fungible_faucet" to access the metadata storage slot of the BasicFungibleFaucet component instead of index 0. In Rust APIs we could use the actual String (or a wrapper type to be precise), but in MASM, due to lack of native strings, we'd have to use a felt-encoded string.

Concretely:

// Currently:
let item = account.storage().get_item(0)?;
// Named Slots:
const SLOT_NAME: SlotName = SlotName::from_static_str("miden.basic_fungible_faucet");
let item = account.storage().get_item(&SLOT_NAME)?;

Motivation

The motivation is that with such an interface, I think we could solve two issues:

• The potential for procedure roots to conflict (see Potential for procedure root conflicts between components #1329). The problem here is that multiple components may likely expose a procedure like push.SLOT_IDX exec.account::get_item which will result in the same procedure root if SLOT_IDX is the same (e.g. 0). That in turn is not allowed in account components because the procedures in those components should have different storage offsets. But since we can only set one offset per MAST root, it must be disallowed.
  • With named storage slots the names of account components would likely be unique (though it cannot be guaranteed or enforced) and therefore the procedure roots would be different.
• The additional complexity introduced by "reserved" slots, like the faucet reserved slot. This creates some additional logic to offset storage indices or deny access to that slot if an account is a faucet, but not otherwise.
  • In the future this may or may not include a storage slot that is reserved on to public or network accounts that describes which note script roots it is willing to accept (see Add NetworkAccount enum #1275 (comment)).
  • Examples for where this requires special handling now:
    • https://github.com/0xMiden/miden-base/blob/b5b3f185701e71861b445567a870f202da6c6a7b/crates/miden-lib/asm/kernels/transaction/lib/account.masm#L422-L426
    • https://github.com/0xMiden/miden-base/blob/b5b3f185701e71861b445567a870f202da6c6a7b/crates/miden-objects/src/account/code/mod.rs#L83-L85
  • With named storage slots we could give reserved slots a unique name and it would work seamlessly with other reserved slots or account components, because all use the same mechanism. Granted, even with named storage slots we still would have to prevent access to the reserved faucet slot, but it is easier to do than with indices and has fewer side-effects. In particular, the logic in the two examples above could go away.

Design

Memory Representation
Storage slots are currently represented in TX kernel memory as:

[STORAGE_SLOT_VALUE, storage_slot_type, 0, 0, 0]

We can repurpose those three zero felts (or even more bits from the felt in which the storage_slot_type is stored) to store an encoded string, similar to the TokenSymbol encoding (but this is to be discussed). One felt can store 13 characters (without encoding the length), so three felts would give us 39 characters, which should be enough. These would effectively add 24 bytes of storage per account slot.

If we add . and _ to the alphabet, then we can represent slightly fewer characters but could then use names like miden.basic_fungible_faucet to identify the storage slot of the basic fungible faucet. Recommending account component authors to namespace their slot names would be a good idea for uniqueness.

All MASM code would use the encoded word as a key to lookup a storage slot. This would result in unique roots for getter-like procedures.

Storage creation
The kernel should guarantee when the account storage is created (i.e. on new accounts) or updated that each name is unique. That then guarantees that each name <-> index mapping is unique in both directions.

Removing storage offsets/sizes
At the same time, I believe we could get rid of AccountProcedureInfo per code procedure (which is 34 bytes) because we would no longer need storage offsets or storage sizes. The primary purpose of these, I think, is to avoid out of bounds access and prevent access from one component's storage to another's. Out of bounds is not really a concern when using names (though we do have to handle lookups of non-existent slot names), and preventing multiple components from accessing the same slot is currently not protocol-enforceable anyway, because storage offsets/sizes are user-defined in the first place.

The advice provider data (CODE_COMMITMENT -> [[ACCOUNT_PROCEDURE_DATA]]) should be extractable from the MastForest in AccountCode and the AccountProcedureIndexMap should work the same (just without storage offsets/size), but I haven't verified this fully yet.

Lookup
The kernel would store the slots like it does now. In order to map a requested name to the index, it would emit an event upon which the transaction host would push the index of the storage slot onto the advice stack. The kernel reads it from there, looks up the slot and verifies that the name of the slot matches the requested one. That way a lookup is not more expensive than it is now.

Deny reserved slot access
It's not clear yet how many reserved slots we'll have or if access to all of them must be denied. But assuming there is more than one such slot, on every access, we could run the following procedure:

#! Inputs:  [SLOT_NAME]
#! Outputs: [SLOT_NAME]
proc.assert_non_reserved_slot_access
	# deny access to faucet reserved slot
	dupw exec.account::get_faucet_storage_data_slot_name eqw not assert

	# deny access to network account note script root slot
	dupw exec.account::get_network_account_note_roots_slot_name eqw not assert

        # ...
end

This is easier to do with names than with indices, because the slots may not always be present in the first place. E.g. for regular accounts we do not have to deny access to slot 0, but we do for faucet accounts. With names, we can deny access by name which does not require additional checks.

In Practice
In practice, for MASM users, I imagine they would choose a storage slot name, use a Rust API to convert that string to its Word representation and then hardcode this word into the MASM code.

For compiler users, I assume they would be able to use the Rust APIs of the SlotName type we would define. That could look something like this for the basic fungible faucet:

const FAUCET_METADATA_SLOT_NAME: Word = SlotName::from_static_str("miden.basic_fungible_faucet").encode();

That would effectively convert the slot name into its Word representation so that users do not have to do any of the manual work.

Conclusion
Upsides:

• Effectively reduces the storage requirements of an account due to getting rid of AccountProcedureInfo.
• No procedure root conflicts for getters.
• Reserved slots and account components work together more seamlessly (but not completely due to the deny requirement)
• No confusion for users about whether multiple components who both access slot index 0 will conflict. Access by name is easy to understand.
• Makes it easier for clients or nodes to access a storage slot of a specific component. For example, the node can request the note roots of a network account from storage by name, e.g. miden.network_account_note_roots instead of having to figure out which storage slot index contains the roots. It doesn't matter whether it is a reserved slot or a component slot.
• Makes account storage easier to reason about (for protocol devs) because one doesn't have to keep in mind that user-accessible storage for faucets starts at 1 but for regular accounts starts at 0.

Downsides:

• MASM interface for get_{map}_item and set_{map}_item is arguably harder to work with as it needs a slot name of type Word rather than just a zero-based index.
  • Example: The set_item signature currently is:
    • Inputs: [index, V']
    • Outputs: [V]
  • and would become
    • Inputs: [SLOT_NAME, V']
    • Outputs: [V]
• Users need to come up with a unique name for each slot and failing to do so could result in incompatible account components.
  • This might not necessarily be a bad thing. With some guidance in docs, this should be doable. But it is harder to work with than a simple index.
• Slightly more complex slot lookup using the transaction host and events.

Alternatives

I thought about whether using the hash of a string like "miden.basic_fungible_faucet" would make sense. That would have the same probability of a collision, but would not be decodable. So for introspection purposes it is much nicer if the name encodes a word which can be reversed and displayed.

[bobbinth]

Very interesting idea! And I think it is very promising. The main concern I have with this is how to make sure slots are indeed named uniquely. We can enforce this at account instantiation time, but not at development time.

For example, I could imagine two components (developed by different developers), pick slot name settings for their components. When we try add these components to the account, there will be a collision, which would basically make these components incompatible.

Maybe this could be alleviated with namespacing rules - e.g., the names would need to be something like <project>.<component>.<slot_name> - e.g., miden.basic_fungible_faucet.faucet_metadata. But names like this will likely go over the 39 char limit (100 chars would probably be fine though).

[bobbinth]

Ah - I should have clarified: the idea here is slightly different: what we do is hash the string and get a Word and then the constant is just a word-sized value. So, there are no restrictions on the name length. Maybe I should have written this as:

const.METADATA_SLOT_NAME=hash("miden::basic_fungible_faucet::metadata")

But we don't do this for error messages - so, I was trying to keep the syntax consistent.

[bitwalker]

Wouldn't a word still be too large to fit in the 3 unused elements of the storage slot? Pre-hashing might have benefits, but the issue is still the space available in the layout AIUI

[bobbinth]

Yes, but I actually don't think we need the full word - collision resistance of a 128-bit value (2 field elements) should be good enough. This is what I was trying to get at with the following MASM syntax:

begin
    # push the first two elements of the word onto the stack
    push.METADATA_SLOT_NAME[0..2]
end

[PhilippGackstatter]

Should we create an issue in the VM for this "hash constant string" approach?

[bobbinth]

Yes, let's do it! I think there are actually two issues:

1. Being able to reference a part of a word-sized constant in push instructions (e.g., push.MY_CONST[0..2]).
2. Declaring word-sized constants from strings (kind of how it already works for error messages).

[PhilippGackstatter]

Just to write this down, one challenge for a block explorer is that it wants to display all token symbols of all public faucets. Currently, only the basic fungible faucet account component provides such a symbol, but there could be other implementations of faucets as well that lay out their storage differently.

With named storage slots we could more easily standardize certain slot names. For example, the storage slot with name miden.faucet.metadata could contain a standardized faucet metadata layout, e.g. token symbol, decimals, etc. Multiple faucets could use that same slot. Right now with indexed storage slots that would be hard to do in practice because one would have to take care that the metadata slot is mapped (during account component merging) to the same global slot index in which the metadata is expected, and which one would that be? It would have to be defined as always being slot X, which is quite likely to produce collisions with other standardization efforts.

Whether the overall idea for sharing faucet metadata is good or not I'm not yet sure about, but the overarching idea for standardization could be one good motivation for named slots.

[PhilippGackstatter]

Shared Storage Slots

After the initial named storage slots PR (#2025) there are different ways in which APIs could work with named storage slots, specifically how to share slots.

Sharing a slot

Let's take a concrete example.

The RPO Falcon512 component and its ACL variant both use a "rpo falcon 512 public key", and so there are two possible setups:

• Store their public keys in different slots, i.e.:
  • miden::standards::auth::rpo_falcon512::public_key
  • miden::standards::auth::rpo_falcon512_acl::public_key
• Use the same slot, i.e. miden::standards::auth::rpo_falcon512::public_key.

The pros and cons of sharing:

• Pro: It would be possible to rewrite procedures like miden::auth::rpo_falcon512::authenticate_transaction to take no parameter instead of the current PUB_KEY parameter. It could instead read the public key from the well-known (hardcoded) slot.
• Con: It could make this procedure less useful for cases where one has a PUB_KEY on the stack, but it's unclear to me if that happens in practice.
• Pro: An imaginary AuthRpoFalcon512KeyRotation component, that implements key rotation, can rotate the key in the well-known slot instead of requiring the slot name to be passed as a parameter. This simplifies interaction between components significantly, since users do not have to extract the slot name from their AuthRpoFalcon512 component and pass it to the rotate component.
  • This makes AuthRpoFalcon512KeyRotation automatically compatible with any component that uses that well-known slot name.

Here, I think the benefits outweigh the downsides and sharing seems like a good idea. It would also be easy to implement sharing to see if there are any unforseen consequences.

Depending on a slot

There are two things to point out about the previous example:

1. AuthRpoFalcon512 and AuthRpoFalcon512Acl share a slot name, but these components cannot be present in the same account (because the protocol only allows exactly one auth procedure), and so sharing the slot name is only relevant to make reusing an API possible (e.g. authenticate_transaction).
2. The rotate component is an example where not only the slot name is shared at the API level, but where multiple components read or write to the same slot.

For the second scenario, I don't think it makes sense for the rotate component to actually contain the public key slot. This would lead to a situation where building an account would look like this:

AccountBuilder::new(...)
  .with_auth_component(AuthRpoFalcon512::new(public_key))
  .with_component(
      AuthRpoFalcon512KeyRotation::new(public_key)
  )
  .build()?;

This creates the base auth component and a rotate component, both with the same slot and the same public key. The account builder would have to merge the two slots (because they have the same name) and ensure their contents match. It is possible, but odd to provide the same parameter twice.

Instead, given that the rotate component is more like an extension of an auth component, it seems better if that account component would not contain an actual storage slot but could express a dependency on it. In this case, the rotate component depends on a the presence of a slot with the name miden::standards::auth::rpo_falcon512::public_key. The AccountBuilder could ensure that there is some component that contains that slot. No merging of slots would be required. This is nice, because the components are only weakly coupled through the slot name, but whether the component that contains the slot is the ACL or the regular component doesn't matter.

This relates to the discussion in #1956, about having a trait that all account components implement. The relevant part of that trait for this functionality could be:

pub trait AccountComponentInterface {
    /// Returns the slot names on which this account component depends.
    /// 
    /// The default implementation returns an empty list.
    fn slot_dependencies(&self) -> Vec<StorageSlotName> {
        vec![]
    }

    // other methods ...
}

Having this would be useful to avoid building an account with a rotate component but without any component that actually contains the storage slot it needs.

Another use case for this pattern is to put getters (for use externally or via FPI) for faucet metadata like get_token_symbol, get_decimals, etc. in a separate component, so they would not have to be duplicated in BasicFungibleFaucet and NetworkFungibleFaucet. This component would therefore depend on the faucet metadata slot, and could be combined with any component that contains that slot. Reducing such duplication is a win in my book.

In general, to make a storage slot of an account A accessible via FPI for an account B, a well-defined interface is needed, and the way to define such an interface could be through such "extension components".

I think this is useful, but I also can't think of enough use cases to be completely sure, yet.

Conversely, I cannot think of a case where two components would contain the same slot and merging them would be required, though if you can think of one, let me know.

Some context that spawned this:

• feat: implement API changes for named storage slots #2025 (comment)
• feat: implement API changes for named storage slots #2025 (comment)

[PhilippGackstatter]

Closing discussion as it was implemented in #1724.