Replies: 5 comments 2 replies
-
|
To continue a bit of the previous discussion in #2349, if you want to use them, I added to EmulatorJS
Combined, these two changes effectively enable polling-based game save update notifications. It's worth noting that the core does write the entire save file to disk every single interval, so probably don't set that value too low. Unfortunately, "too low" is likely dependent upon client hardware, save file size (which I believe can even vary by game for some systems), browser configurations, etc.—but my performance tests suggest that it also doesn't matter too much. However, as I didn't thoroughly test core-by-core, I'll restrain myself from offering a concrete suggestion. I would recommend refining it gradually. I doubt I'll have time to put in more actual work, but I will try to give a brief design review of this RFC later today. I'm not that fussed about how you do things, but want to put a second pair of eyes on 1) protecting from data loss (including conflict resolution), and 2) ensuring contributors' need to do as little work as possible for robustness. |
Beta Was this translation helpful? Give feedback.
-
|
I did spend quite a lot of time reviewing, but ultimately, I think this is good. I only have two pieces of feedback. First, that each additional mode you've described is an incremental improvement. (Or, in any case, you don't need to support all of these features in an initial implementation.) Second, with some small modifications, I think you can slightly reduce the complexity of the system. Although this is an elaborate reply, I don't feel strongly about this. Adopting the server-side save folder structure you outline in the file transfer mode for all cases (since it is the most flexible client-side), and having the orchestrator operate on those save folders, does three things:
Focusing on the first question first, the API mode becomes a simple save upload & download to & from the appropriate folder, created/registered automatically with RomM if necessary. The push case is more challenging. If the clients that support SSH do not support the API, you'll have to compare. (Much, much better would be rsync, since that has rich options that make this easy—but less available than SSH.) Frankly I'm skeptical that this scenario is likely in a way that's more than an optimization—if the client supports pulling saves via API, it definitely is. The second question is trickier, but I dove into it a bit: In my estimation, the primary concerns around conflict resolution come from A) modifications to saves after they are deleted, and B) conflicting modifications to saves. In the interest of preserving data, (A) should undelete the save. (B) is more interesting, but I can only see three ways of resolving this (though perhaps you can see more):
The first option does not require any new negotiation tactics with the client or placement of data indicating a conflict into the save file of the client device. As you said, "device timestamps are not reliable," but I think that this is not as big of a deal as it seems. I could only think of three issues with this approach:
I don't think the latter two are worth considering. The first one cannot be meaningfully resolved by RomM, even with rich metadata—a decision must be made. A third party sync solution will do the same. The point of all this is to say that there's likely nothing to be gained from the rich metadata you could provide via API conflict resolution in practice, and the issues doing so would resolve in API mode would still matter for the other sync modes. To conclude, you can probably save a small amount of time and complexity doing it this way without loss of quality or flexibility. |
Beta Was this translation helpful? Give feedback.
-
|
I want to keep this brief but could I suggest another layer to the saves folder structure. The reason here being that some emulators or games will require the name of the save to be something specific such as the name of the rom or the word "main". A common Solution I've seen to this issue is similar to the solution shown below, where each game will have it's own save folder where all the save data for that game goes. Note: In this example I swapped the game Id number out for the actual name of the game. A more practical solution might be to combine the two whenever a scan is performed. Personally I think this is something that should be implemented even before the [Save Synchronization System]. |
Beta Was this translation helpful? Give feedback.
-
|
Something to think about - you may want to consider API versioning; the /api/ URL is fine, but if you decide to redesign the API, you may want to consider using /api/v1/xxxx and alias it back to /api so that you can update your API without making breaking changes. |
Beta Was this translation helpful? Give feedback.
-
|
One question, and probably shouldn't be relevant for a first edition, but could part of save file conversion be changing the ps user id? ex: migrating between an old ps3 to a ps4 with different accounts and from ps4 to ps5, with ps4 being offline, and ps5 being online enabled and loading saves from usb |
Beta Was this translation helpful? Give feedback.
-
|
No that's outside the scope of RomM, we'll only handle saves based on the internal user ID, not the PS user ID. |
Beta Was this translation helpful? Give feedback.
-
|
ah, gotcha, and tbh fair |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
RFC Number: 0001
Title: Device Registration and Multi-Protocol Save Synchronization
Author: @gantoine
Status: Draft
Created: 2025-08-04
Updated: 2025-08-04
Abstract
This RFC proposes a save synchronization system where clients register with the server using pre-built yet configurable profiles and receive a unique device ID. The system maintains sync state tracking to enable seamless cross-device gameplay continuation, and supports three sync modes: API, file transfer, and push-pull.
Motivation
Synchronization
Cross-device save sync is a an ambitious goal, but one that is fundamental to the success of this project. The expanding ecosystem of client applications demands the ability to pick-up-and-play on any device with seamless gameplay continuation, and existing solutions are inadequate.
Device Registration
As each system has different technical capabilities, a one-size-fits-all approach won't work. A device registration method allows for platform-specific sync mode selection, save format awareness and conversion (file structure, naming, emulator), and flexible sync scheduling based on device capabilities.
Architecture Overview
The save synchronization system consists of three main components: the server-side sync orchestrator, client-side sync agents, and a device registry. The architecture is designed to handle the diverse technical capabilities of different client platforms.
Device Registry Service
Clients register with platform-specific capabilities and receive a unique device ID. Pre-built profiles are available for supported platforms (Playnite, muOS, Android, etc.), but options can be customized.
Sync Orchestrator
The orchestrator tracks the state (last sync time, sync status, etc.) of each device and schedules sync jobs based on device capabilities.
The orchestrator handles conflict resolution naively by always favoring the most recent save. Since device timestamps are not reliable, an alternative conflict resolution strategy is required. In file transfer and push-pull modes, sync state metadata can be pushed to the device and updated on each sync for better conflict resolution, while in API mode, clients handle storing their own metadata.
The orchestrator will prefer devices that use API mode, as it's the most reliable way to sync saves, but will support file transfer and push-pull modes for devices that don't have native client apps.
Sync Engine
API Mode
In API mode, clients negotiate the sync state with the orchestrator and then perform parallel save operations. Clients track which saves have been created/updated and send the list to the orchestrator, which returns operation details for execution. In essence, the client sends the state-of-saves to the orchestrator, and the orchestrator returns the operations to perform.
Device Registration:
{ "device_name": "Anbernic RG40XXV", "platform": "muos", "sync_mode": "api", } { "device_id": "83372e3b-3b30-48a9-a585-40cd1a4da810" }Sync Negotiation:
{ "device_id": "83372e3b-3b30-48a9-a585-40cd1a4da810", "device_time": "2025-08-04T12:00:00Z", "client_state": { "last_sync_time": "2025-08-04T10:30:00Z", "saves": { "created": [ { "operation_id": "op-1", "rom_id": 8273, "filename": "Advance Wars (1).srm", "sha1": "ebd8b595cdc30873a0d17476e396571463fdb731", "file_size": 24785, "updated_at": "2025-08-04T11:00:00Z" } ], "modified": [ { "operation_id": "op-2", "save_id": 283, "filename": "Advance Wars.srm", "sha1": "5123655a27403c1a0697b3583b530c70d16862d0", "file_size": 1024000, "updated_at": "2025-08-04T11:00:00Z" }, { "operation_id": "op-3", "save_id": 1238, "filename": "Crash Nitro Kart (2).srm", "sha1": "7830537639ecd86d6419f365d67aede697978893", "file_size": 8276943, "updated_at": "2025-08-04T11:00:00Z" } ], "deleted": [ { "operation_id": "op-4", "save_id": 7345, "filename": "Jet Grind Radio (3).srm" } ] }, "states": {} } } { "sync_session_id": "f32d6ef4-b49a-4634-a2d5-664a5c32841f", "server_time": "2025-08-04T12:00:00Z", "operations": { "upload": [ { "operation_id": "op-1", "save_id": 8273, "upload_url": "/api/saves/8273?sync_session=f32d6ef4&op_id=op-1" }, { "operation_id": "op-2", "save_id": 283, "upload_url": "/api/saves/283?sync_session=f32d6ef4&op_id=op-2" } ], "download": [ { "operation_id": "op-3", "save_id": 1238, "download_url": "/api/saves/1238?sync_session=f32d6ef4&op_id=op-4" } ], "delete": [ { "operation_id": "op-4", "save_id": 7345 }, { "operation_id": "op-5", "save_id": 9274, } ] } }Clients execute operations in parallel using the provided URLs, where each save endpoint accepts sync context parameters:
The server tracks operation completion within the sync session and handles partial failures gracefully. The success/failure of each operation is handled by the HTTP response codes, and the client updates their own sync state metadata. Note that while save uploads/updates are handled by the client, save deletion is handled by the server.
File Transfer Mode
File Transfer mode is designed for devices that use external file synchronization systems like Syncthing, Resilio Sync, or cloud storage folders. In this mode, RomM monitors designated sync folders and automatically processes save files as they appear.
Devices register with a
file_transfersync mode and specify their sync folder path. The external sync system maintains the device's save folder structure as-is. The server monitors these folders and handles all processing internally.Device Registration:
Folder Structure:
romm_sync/ ├── device_83372e3b-3b30-48a9-a585-40cd1a4da810/ │ ├── gba/ │ │ ├── Advance Wars (USA) (Rev 1).srm │ │ └── Pokemon Fire Red.srm │ ├── psx/ │ │ └── Crash Bandicoot.srm │ └── n64/ │ └── Super Mario 64.srm └── device_f47ac10b-58cc-4372-a567-0e02b2c3d479/ ├── gba/ │ └── Pokemon Fire Red.srm └── psx/ └── Crash Bandicoot.srmWhen a file is modified, the orchestrator compares the file's timestamp and hash against the last known state. If the modification is new, the orchestrator will update the RomM database with the new save data.
When a save is created/updated/deleted by another device (or the system), the orchestrator will modify the sync folder to reflect the change. The orchestrator will not attempt to resolve conflicts as external sync systems handle their own conflict resolution.
This mode is only available for devices that are recognized by the system, i.e. where the folder structure and external sync system is known.
Push-Pull Mode
Push-Pull mode enables server-initiated synchronization using SFTP/SSH connections. The server actively connects to devices, scans their save directories, and performs bidirectional sync operations. This mode is ideal for devices that can't run RomM clients but support SSH access.
Device Registration:
{ "device_name": "RetroPie", "platform": "linux", "sync_mode": "push_pull", "ssh_config": { "host": "192.168.1.100", "port": 22, "username": "pi", "auth_method": "key", "private_key_path": "/romm/ssh/retropie_key", "save_paths": { "gba": "/home/pi/RetroPie/roms/gba/saves", "psx": "/home/pi/RetroPie/roms/psx/memcards", "n64": "/home/pi/RetroPie/roms/n64/saves" } }, "save_formats": { "gba": { "extension": ".srm", "emulator": "mGBA" }, "psx": { "extension": ".srm", "emulator": "PCSX-ReARMed" } } }The server establishes SSH connection using stored credentials and scans configured save directories for all save files. The orchestrator determines which files need to be pulled, pushed, or deleted and performs SFTP operations to sync files bidirectionally. Finally the orchestrator updates the internal tracking and RomM database.
File Discovery:
Sync Operations:
System Diagram
Device Registration and Multi-Protocol Save Synchronization -2025-08-04.json
Implementation Details
Database Schema
API Endpoints
POST /api/devices- Device registrationGET /api/devices/{device_id}- Get device infoPUT /api/devices/{device_id}- Update device configDELETE /api/devices/{device_id}- Deregister devicePOST /api/sync/negotiate- Sync negotiation (API mode)GET /api/sync/status- Get sync statusPOST /api/saves/{save_id}- Upload save fileGET /api/saves/{save_id}- Download save fileDELETE /api/saves/{save_id}- Delete save fileConflict Resolution Strategy
Detection Methods
Resolution Policies
Conflict Notification
When conflicts are detected in FTM or PPM, the orchestrator will display a list of conflicts in the web app. The user can then choose to resolve the conflicts manually or automatically. In API mode, the orchestrator will return a list of conflicts to resolve and the client will handle the resolution.
Sync Frequency
Metrics
Beta Was this translation helpful? Give feedback.
All reactions