NIP-17: "seen" event

[fiatjaf]

A simpler alternative to #1761.

Here's an image with the diff for your delight:

[kehiy]

Good. Can we cover other chat specific statuses in the same way? One more people accepted this, we can close the older PR. Keeping PRs clean.

[kehiy]

cc @vitorpamplona @wcat7 @erskingardner @Giszmo

[reyamir]

I think this will create an issue. If some bad actors decide to attack another client by sending a seen event that includes a large number of message ids, the other client may become stuck because it has to check so many event ids.

[fiatjaf]

What do you mean by "check"? If you have the messages in memory already that should be trivial, no? Otherwise you can ignore that, or only check the last ones.

[reyamir]

What do you mean by "check"? If you have the messages in memory already that should be trivial, no? Otherwise you can ignore that, or only check the last ones.

Ah okay, I understand it now, I'm a bit is overthinking.

[kehiy]

Would this work with huge chats? Consider this case:

I have a huge chat with my friend in 0xChat and we have seen events for them too. Then I open Coop on my desktop for the first time. The Coops event storage is empty. it needs to get synced with both messages event and seen events and match them too (We need to be able to query them well also). But if we keep one universal event in a way that it holds a bloom filter or a timestamp, we query 1 event per chat and we indicate seen/unseen statues immediately. since it's hard to find all events on the Nostr and we are even struggling with this issue with the messages themselves, fetching all seen events per chat per message may lead to an even worse experience.

Also, Is this one works on groups?

[reyamir]

Would this work with huge chats? Consider this case:

I have a huge chat with my friend in 0xChat and we have seen events for them too. Then I open Coop on my desktop for the first time. The Coops event storage is empty. it needs to get synced with both messages event and seen events and match them too (We need to be able to query them well also). But if we keep one universal event in a way that it holds a bloom filter or a timestamp, we query 1 event per chat and we indicate seen/unseen statues immediately. since it's hard to find all events on the Nostr and we are even struggling with this issue with the messages themselves, fetching all seen events per chat per message may lead to an even worse experience.

Also, Is this one works on groups?

The process is simpler than you think.

→ Open Coop
→ Open a chat panel
→ Coop loads all messages / Coop fetches the 'seen' event
→ Coop extracts all message ids from that 'seen' event into Vec<EventId>
→ when rendering a message, Coop checks whether the message ID is present in that Vec or its timestamp is earlier than the last item’s. If so, it shows the seen mark.

No query is needed.

[vitorpamplona]

1. What happens if multiple chats have the same subject? And doesn't that break the privacy of chat group subjects?
2. What if, instead of the IDs of the posts which can be used to track messages, we just did "seen" and one "timestamp"? Everything is seen until that point.
3. This will generate LOTs of extra traffic (encryptions, decryptions to each recipient).
4. Why do we insist on designs that use [tag name, value, value, value] instead of [tag name, value], [tag name, value], [tag name, value]?
5. Do we really want this? Or do we just want to know if the user has been online with a NIP-17 client to see the message?

[fiatjaf]

1. These events are gift-wrapped and sent like normal messages. I actually don't understand the "subject" stuff but I assume it's used to differentiate two messages with the same pair of (sender, receiver), in this case they just serve the same goal here.
2. I wrote it like that first, but then I thought the "seen" could also be used to track missed messages in a short way (I know it's not the best or most reliable, but seemed simple and good enough for these chats that no one cares much about in the end).
3. Because it's a list of things, they won't be queried or anything, seemed better this way. Why is it bad? Also a simpler client could opt to just read the first "id" and assume everything up to that has been seen and ignore the rest.

[vitorpamplona]

How about this:

"Received" event

A public (not giftwrapped) event of kind 10017 MUST be updated every time the NIP-17 client decrypts a new message. This update will mark messages until this point as "received" and confirm to your friends that you are still using a NIP-17 client and can see their messages.

{
  "id": "<usual hash>",
  "pubkey": "<pubkey>",
  "created_at": "<now>",
  "kind": 10017,
  "tags": [],
  "content": ""
}

[staab]

Tracking individual event ids gets really hairy, speaking from experience. Timestamp-based tracking is better, and should only lie if relay selections aren't implemented properly. I'm not sure Vitor's ultra-simple thing is sufficient, depending on use case, since as @kehiy said we might want stuff like "typing...".

[vitorpamplona]

There are 3 things here:

• Mark as seen for yourself, so that your clients can sync with each other. @kehiy's Bloom filter on a 10xxx event is possibly the best idea.
• Mark as seen for friends, kind 10017 seems to be the easiest idea to avoid tons of data and memory issues
• "Typing...", which can be a public event as well:

"Typing" event

A public (not giftwrapped) event of kind 10018 may be used to indicate when the user is typing into a chatroom.

{
  "id": "<usual hash>",
  "pubkey": "<pubkey>",
  "created_at": "<now>",
  "kind": 10018,
  "tags": [
     ["room", "100:10:AAAkAQANcYQFCQoB:hZkZYqqdxcE="]
     ["expiration", "now + 15"]
  ],
  "content": ""
}

Where room is Bloom filter of the IDs of the last messages of the room as defined on: https://github.com/nostr-protocol/nips/pull/1497/files

Clients should download kind 10018 and check if the ID of the 3 most recent kind 14 messages are inside the filter.

[staab]

Bloom filters could be ok for sharing data with others, but would result in really annoying non-dismissable notifications 1% of the time for users. I think the best approach for that is going to be timestamps checked against when an event was first seen by the client (rather than its created at).

[vitorpamplona]

would result in really annoying non-dismissable notifications 1% of the time for users

humm.. can you explain more? Why are they non-dismissable? They don't even p-tag receivers. I am not sure where these notifications are coming from. You would only track 10017 and 10018 for users in in the room.

[staab]

Sorry, I was getting the bloom filters false positive/negative mixed up. If you're keeping a bloom filter with event ids, you'll sometimes get false positives, resulting in missed notifications. Also, the bloom filter will grow forever, so there needs to be some timestamp-based rule for removing events from the filter.

[vitorpamplona]

For the typing event, we can just store 3 or so of the last IDs. There is no need to store long term.

For the Mark as seen for yourself event, we could just store the latest ID per chatroom (group of p tags)

[fiatjaf]

I think you're all crazy, bloom filters are obviously an overkill for this. Why would you want to check if a message from 6 months ago was read? You only care about like the last 5 messages.

[fiatjaf]

Just the timestamps will suffice definitely, but this proposal's ability to include some ids (again, not all ids, that would be crazy, please read my proposal) is a small enhancement over that to account for messages that are late to arrive, messages that get lost, multiple messages that have the same timestamp etc.

[staab]

not all ids, that would be crazy, please read my proposal

I'm just saying I tried this, and it was still crazy. If you're including recent ids, you need to set the timestamp to the time the event before the last id you include was created. I can't remember what the exact problem was, but there was some other syncing gotcha. Timestamps based on when an event was seen (for the user) and when an event was created (for their counterparty) are an order of magnitude simpler.

[vitorpamplona]

I think you're all crazy, bloom filters are obviously an overkill for this. Why would you want to check if a message from 6 months ago was read? You only care about like the last 5 messages.

The bloom filters are public so that you don't need to encrypt/decrypt to every recipient (HUGE data consumption). They are also very small since it is. only the last 3 Ids or so. The filter just obfuscates the IDs for a public event.

[vitorpamplona]

Keep in mind that even though you are transferring 30016 inside GiftWraps, the wraps/seals are not replaceable, which means that if the client doesn't add a reasonable expiration in the wrap event, in the long run we will have more giftwraps with old seen events than anything else.

[fiatjaf]

The bloom filters are public so that you don't need to encrypt/decrypt to every recipient (HUGE data consumption). They are also very small since it is. only the last 3 Ids or so. The filter just obfuscates the IDs for a public event.

I see. That means I didn't read the other proposal then.

Keep in mind that even though you are transferring 30016 inside GiftWraps, the wraps/seals are not replaceable, which means that if the client doesn't add a reasonable expiration in the wrap event, in the long run we will have more giftwraps with old seen events than anything else.

Good point. Maybe giftwraps were really a bad idea, they are very hostile to things like relay disk and memory, spam protection etc. We should come up with a different protocol.

[vitorpamplona]

Maybe giftwraps were really a bad idea, they are very hostile to things like relay disk and memory, spam protection etc.

They are not designed for mundane things like "Mark as Seen". They are designed for a metadata-leak-free privacy. We don't need to apply that crazy level of privacy for most things on Nostr.

[wcat7]

A small idea: make the seen tag an optional field inside kind 14 / kind 15 events.