Challenge 4 - Scaffolding & Signals

In this challenge, we will build a messaging app. The frontend UI has been provided for you, however the backend has not. Here is an outline of the steps and lessons in this challenge:

• 1. Scaffolding entry types, collections, and links
• 2. Implementing zome functions
• 3. Sending and receiving remote signals from other Agents
• 4. Customizing remote signals

Setup

1. Fork this repo and clone it onto your local machine

2. cd into holochain-challenge-4 directory and run nix develop.

3. Run npm i to install package dependencies

Scaffolding entry types, collections, and links

When you use the scaffolding tool, Holochain scaffolds both the backend and the frontend. However for this challenge, the UI has been provided for you inside ui/src/chatroom/premade-chatroom-ui as we will be primarily focusing on the rust backend. You can still expect files generated by the scaffolding tool to show up inside the chatroom folder, however these can be ignored.

Before we get started there are a few key things to note:

• Everything you scaffold should be spelled as shown in the instructions. This is so that our Svelte frontend can interact with your Rust code correctly.
• If you make a mistake half way through a scaffold, in the terminal press Ctrl + C to cancel it, and start again.
• After each scaffold, I'd recommend you look through the generated code to understand and familiarize yourself with it.
• I'd also recommend you make a new commit to your git repo after each scaffold.
• npm start won't work until we've finished scaffolding

1. Make sure that your project contains a dnas folder at its root, if it doesn't, then create one

2. Navigate to the terminal, and run hc scaffold dna

Provide the following values for the prompts:

• DNA name: chatroom

3. Navigate to the terminal, and run hc scaffold zome

• What do you want to scaffold? Integrity/coordinator zome-pair
• Enter coordinator zome name: chatroom
• Scaffold integrity zome in folder "dnas/chatroom/zomes/integrity/"? y
• Scaffold coordinator zome in "dnas/chatroom/zomes/coordinator/"? y

Tip!

After each scaffold, you should read through the generated content to more deeply understand what the scaffold is actually creating. A lot of it will look familiar from earlier challenges.

4. Run hc scaffold entry-type

• Entry type name: room
• Field name: name
• Which fields should the entry contain? String
• Should this field be visible in the UI? n
  • Normally you'd choose yes, but in this instance we've created the frontend for you
• Add another field to the entry? y
• Field name: creator
• Choose field type: AgentPubKey
• Should a link from the AgentPubKey provided in this field also be created when entries of this type are created? y
  • Choosing this option will mean a link is created from the Creator Agent to the room in the DHT
• Which role does this agent play in the relationship? creator
• Field name: creator
• Add another field to the entry? n
• Confirm the fields are correct and hit 'Confirm'
• Which CRUD functions should be scaffolded? - Untick both Update, and Delete options

5. Run hc scaffold collection

• Collection name: all_rooms
• Which type of collection should be scaffolded? Global
• Which entry type should be collected? Room

6. Run hc scaffold entry-type

• Entry type name: message

• Field name: content

• Choose field type: String

• Should this field be visible in the UI? n

• Add another field to the entry? y

• Choose field type: AgentPubKey

• Should a link from the AgentPubKey provided in this field also be created when entries of this type are created? y

• Which role does this agent play in the relationship? creator

• Field name: creator

• Add another field to the entry? y

• Field name: timestamp

• Choose field type: Timestamp

• Should this field be visible in the UI? n

• Add another field to the entry? y

• Field name: room_hash

• Choose field type: ActionHash

• Should a link from the ActionHash provided in this field also be created when entries of this type are created? y

  • Choosing yes will create a link from this action hash (the room_hash) to the message, when a message is being created

• Which entry type is this field referring to? Room

• Add another field to the entry? n

• Confirm the fields are correct and hit 'Confirm'

• Which CRUD functions should be scaffolded? - Untick both Update, and Delete options

7. Run hc scaffold link-type

• Link from which entry type? Agent
• Which role does this agent play in the relationship? member
• Link to which entry type? Room
• Reference this entry type with its entry hash or its action hash? ActionHash
• Should the link be bidirectional? y
  • Choosing y for this option creates two links. One from each Member Agent to each Room, and one from each Room to each Member. We want this because these entries have a many to many relationship (One room can have many members, one member can be apart of many rooms)
• Can the link be deleted? n

Good job! You've scaffolded everything you need to for this project!

Implementing zome functions

Now that we have scaffolded out the bulk of the project, we just need to implement a couple of changes to our zome functions that are more specific to this app, and aren't supplied by the scaffolder.

1. Start by running npm start and opening up an agent window

You will notice our client is trying to make a zome call to fetch the available chatrooms, but is running into an error.

2. Navigate to chatroom/premade-chatroom-ui/SearchRooms.svelte

This is where the failing zome call, get_not_joined_rooms_for_member is coming from.

Function signature hint

pub fn get_not_joined_rooms_for_member(member: AgentPubKey) -> ExternResult<Vec<Link>>

Which rust file should this code go in?

3. Navigate to dnas/chatroom/zomes/coordinator/chatroom/src/member_to_rooms.rs and implement a get_not_joined_rooms_for_member zome function that returns ExternResult<Vec<Link>>

// Implementing getNotJointRoomsForMember function:
// Step 1: Get all the rooms from the DHT using the all_rooms path
// Step 2: Get all rooms the member has already joined
// Step 3: Convert the joined rooms to a HashSet for efficient lookups
// Step 4: Filter the all_rooms list to exclude the rooms the member has already joined
// Step 5: Return the filtered list of rooms that the member hasn't joined yet

Hint - Breakdown of each step

#[hdk_extern]
pub fn get_not_joined_rooms_for_member(member: AgentPubKey) -> ExternResult<Vec<Link>> {
    let path = Path::from("all_rooms");
    let all_rooms = get_links(
        GetLinksInputBuilder::try_new(path.path_entry_hash()?, LinkTypes::AllRooms)?.build()
    )?;
    let joined_rooms = get_links(
        GetLinksInputBuilder::try_new(member, LinkTypes::MemberToRooms)?.build()
    )?;
    let joined_rooms_set: HashSet<_> = joined_rooms
        .into_iter()
        .map(|r| r.target)
        .collect();
    let not_joined_rooms = all_rooms
        .into_iter()
        .filter(|room| !joined_rooms_set.contains(&room.target))
        .collect::<Vec<_>>();
    Ok(not_joined_rooms)
}

Now when you create a chatroom, different agents can join it.

Don't forget restart your server as we've changed some zome code.

4. Have an agent auto join a room when they create it.

Navigate to coordinator/chatroom/src/room.rs and modify the create_room function to also create two links: RoomToMembers and MemberToRooms

This means that when an agent creates a new room, they will also 'join' it as a member.

Hint

    create_link(room.creator.clone(), room_hash.clone(), LinkTypes::MemberToRooms, ())?;
    create_link(room_hash.clone(), room.creator.clone(), LinkTypes::RoomToMembers, ())?;

The behaviour change will be subtle but important. Now when a room is created and you click on the joined rooms icon in the top right of the UI the room will show up.

Sending and receiving remote signals from other Agents

In our app we can create chatrooms, join them, and send messages to other Agents inside them. However every time a chatroom is created or a new message is sent, we have to manually refresh the window to view the new changes.

Try and create and join rooms from both your agents and chat with each other. You'll need to refresh the page a lot, which is lame. Let's make it better.

In this section we will use remote signals create a live update on each client when a new message is sent to a chatroom.

1. Navigate to the signal_action function inside of coordinator/chatroom/src/lib.rs

This function is responsible for emitting local signals to the client for each type of action that occurs.

Inside the Action::Create arm of the match statement, we can see that we are emitting local signals to our client on each execution of a create action.

Modify this code so that when a create action containing a Message entry is made, we also emit a remote signal to each agent in the chat room.

The code should be along the lines of this:

Action::Create(_create) => {
    if let Ok(Some(app_entry)) = get_entry_for_action(&action.hashed.hash) {
        let new_signal = Signal::EntryCreated {
            app_entry: app_entry.clone(),
            action: action.clone(),
        };
        emit_signal(&new_signal)?;

        // If the create action's entry is of type Message

        // Get the entry of the create action

        // Get the room hash from the entry

        // call get_members_for_room() to get the members for the room using the room hash

        // Filter the members vector to exclude our agent pubkey (we don't want to send a remote signal to ourself)

        // remote_signal(new_signal, members)
    }
    Ok(())
}

This is a little different to the scaffolded code as we've extracted out a new_signal variable so we can easily reuse it for the remote signal.

To access the get_members_for_room function you will need the following at the top of your file

use crate::member_to_rooms::get_members_for_room;

Hint

Action::Create(_create) => {
    if let Ok(Some(app_entry)) = get_entry_for_action(&action.hashed.hash) {
        let new_signal = Signal::EntryCreated {
            app_entry,
            action: action.clone(),
        };
        emit_signal(&new_signal)?;

        // If the create action is of type Message
        if
            action.action().entry_type().unwrap().clone() ==
            UnitEntryTypes::Message.try_into()?
        {
            // Get the entry off the create action
            let record = get(action.hashed.hash.clone(), GetOptions::default())?.unwrap();
            let message = record.entry().to_app_option::<Message>().unwrap().unwrap();

            // Get the room hash from the entry
            let room_hash = message.room_hash;

            // Get the members for the room using the room hash
            let members: Vec<AgentPubKey> = get_members_for_room(room_hash.clone())?
                .into_iter()
                .map(|link| {
                    AgentPubKey::try_from(link.target)
                        .map_err(|_| {
                            wasm_error!(
                                WasmErrorInner::Guest(
                                    String::from("Could not convert link to agent pub key")
                                )
                            )
                        })
                        .unwrap()
                })
                .filter(|agent| *agent != agent_info().unwrap().agent_latest_pubkey)
                .collect();

            let _ = send_remote_signal(new_signal, members);
        }
    }
    Ok(())
}

2. Paste the following code snippet inside coordinator/chatroom/src/lib.rs

#[hdk_extern]
fn recv_remote_signal(signal: Signal) -> ExternResult<()> {
    emit_signal(signal)?;
    Ok(())
}

This function will receive the remote signal from the network and forward it to the client's frontend.

3. Replace the current init function inside coordinator/chatroom/src/lib.rs with the following code.

To allow other agents on the network to directly call the recv_remote_signal function, we need to give them access with a Capibility Grant. We will cover this more in depth in a future challenge.

#[hdk_extern]
pub fn init(_: ()) -> ExternResult<InitCallbackResult> {
    let mut functions: BTreeSet<(ZomeName, FunctionName)> = BTreeSet::new();
    functions.insert((ZomeName::from("chatroom"), FunctionName::from("recv_remote_signal")));
    create_cap_grant(CapGrantEntry {
        tag: "recv_remote_signal_unrestricted".into(),
        access: CapAccess::Unrestricted,
        functions: GrantedFunctions::Listed(functions),
    })?;
    Ok(InitCallbackResult::Pass)
}

4. Save the file, and restart your server.

Now when you send a message, it is instantly received by the other client! - Kind of!

5. Navigate to the onMount function of premade-chatroom-ui/Conversation.svelte

Inside client.on() Notice how when we receive a Message signal, we are instantly adding its hash to a 'hashes' array. This will cause svelte to re-render, and create a new Message Component, which will retrieve and display the Message record from the DHT by this hash.

The issue is that Signals are faster than DHT propoagation! The remote signal is being received and the new message is requested from the DHT before the DHT has properly registered the addition of the new message.

We have access to the full entry content in payload.app_entry so why not just display that instead of forcing the Message component to load the hash contents?

6. Fix the issue of messages being undefined

This issue isn't just about timing - it's actually an important security consideration.

When we receive a signal that a new message was created, we shouldn't trust the signal content itself. Remember any agent in our network can send a signal and it will be blindly fowarded to our client. A malicious agent could potentially craft signals claiming to be from anyone. Instead, we should:

1. Receive the signal that contains the hash of the new message
2. Request the actual message record from the DHT
3. By doing this, the message goes through proper validation and we know for sure it was actually created by the claimed author.

Unfortunately, there's a race condition here - our code tries to fetch the message before it's fully committed to the DHT. To solve this, we need to implement a retry mechanism in the Message.svelte component:

// At the top of your <script> section in Message.svelte
const INITIAL_RETRY_DELAY = 1000; // 1 second
const MAX_RETRIES = 5;
let retryCount = 0;
let retryTimeout: NodeJS.Timeout | undefined;

// Then modify your fetchMessage function to include retries
async function fetchMessage() {
  loading = true
  error = undefined

  try {
    record = await client.callZome({
      cap_secret: null,
      role_name: 'chatroom',
      zome_name: 'chatroom',
      fn_name: 'get_message',
      payload: messageHash,
    })
    
    if (record) {
      message = decode((record.entry as any).Present.entry) as Message
      messageCreator = encodeHashToBase64(message?.creator)
      messageCreatorSliced = messageCreator.slice(0, 7)
      retryCount = 0; // Reset retry count on success
    } else {
      message = undefined;
      if (retryCount < MAX_RETRIES) {
        const delay = INITIAL_RETRY_DELAY * Math.pow(2, retryCount);
        retryCount++;
        console.log(`Message not found, retrying in ${delay}ms (attempt ${retryCount}/${MAX_RETRIES})`);
        retryTimeout = setTimeout(fetchMessage, delay);
      } else {
        console.log('Max retries reached, giving up');
        error = new Error('Failed to load message after maximum retries');
      }
    }
  } catch (e) {
    console.log(e);
    error = e;
    if (retryCount < MAX_RETRIES) {
      const delay = INITIAL_RETRY_DELAY * Math.pow(2, retryCount);
      retryCount++;
      console.log(`Error fetching message, retrying in ${delay}ms (attempt ${retryCount}/${MAX_RETRIES})`);
      retryTimeout = setTimeout(fetchMessage, delay);
    }
  }

  loading = false
}

// Don't forget to clean up the timeout in onMount:
onMount(async () => {
  // ... your existing code ...
  
  // Return a cleanup function
  return () => {
    if (retryTimeout) {
      clearTimeout(retryTimeout);
    }
  };
})

You may also want to display loading instead of undefined

    <span style="white-space: pre-line">{message?.content || 'Loading...'} </span>

This approach provides several benefits:

1. Security: We're fetching the validated message from the DHT rather than trusting the signal content
2. Reliability: The exponential backoff retry mechanism gives more time for the DHT to fully process the message
3. User Experience: Multiple retry attempts with increasing delays provide the best chance of success
4. Error Handling: We show proper error messages if all retries fail

The key security point is that we never trust the signal content directly - we always fetch and validate the actual message from the DHT.

Thats a wrap! 👏