-
Notifications
You must be signed in to change notification settings - Fork 10
How Sync Works
As of v0.10.2
In order to make sure client & server versions are compatible, the first step taken is to check the ModSync server version. If the client and server differ by at least one minor version, the server will enter Rescue Mode for that client.
Next, the client will request info on all of the paths the server is requesting to sync. This determines which files the client will hash and compare later on.
As an intermediary step, the client will check what version of ModSync was previously used to perform a sync. If it finds an older version it will attempt to update any previously saved configuration files and the PreviousSync.json
The PreviousSync.json file keeps track of what files were previously reported by the server. This information is used by the client for non-enforced sync paths to enable some flexibility in changing local files without the server immediately overriding them. When enforced is true slightly different logic is used for checking modified files.
At this stage, the JSON file is loaded, if it exists.
Then, the BepInEx/config/corter.modsync.cfg and local exclusions are loaded from ModSync_Data/Exclusions.json.
Exclusions set on the server are requested and cached by the client.
Next, starting with the provided sync paths, the client will recursively search for files. Both server exclusions and local exclusions are taken into account while search and matching files or folders will be ignored (files are not hashed and children are ignored).
Several popular mods ship with empty folders and will crash if they are not present. Because of this ModSync has special handling for empty folders and will sync them as if they were files.
Hashing is an integral of how ModSync works. Generated hashes are used to compare files from the server and client without manually comparing each byte of the file.
For each discovered file ModSync computes the file's hash. The exact way this occurs depends on the size of the file being hashed.
- Files
< 10MBare read straight into memory and hashed in their entirety using the MetroHash algorithm[^1][^2]. - Files
> 10MB(usually bundles) are hashed using a really interesting "constant-time hashing algorithm" called imohash. imohash works by sampling 32KB chunks of the file at the beginning, middle, and end. This hash is augmented with the size of file to form a heuristic hash.
Now the client requests file hashes for enabled sync paths to compare.
The comparison is the heart of ModSync, the secret sauce, if you will. By comparing the list of files and hashes from the server with the one the client generates, ModSync can determine
which files have been added, updated, removed, and empty directories that have been created.
Perhaps the simplest comparison done by ModSync is added files. To determine added files, ModSync finds file entries on the server that aren't present on the client.
To determine updated files, ModSync filters for files that are present on both the client and the server. The server hashes for these files are compared to those found in PreviousSync.json. If the previous sync and the server hashes don't match then the client and server hashes are compared. If these hashes don't match then, and only then, the files are considered updated and queued for download.
A file will be considered removed when it was previously synced (therefore is present in the PreviousSync.json) but no longer present on the server. This is done as a safety net against deleting files that get created on the client, but are not present on the server.
Another simple comparison, created directories are found by comparing folders present on the server that aren't present on the client.
During this stage, ModSync will download any added/updated files in parallel batches of 8 files. These files are downloaded into the ModSync_Data/PendingUpdates directory with the notable exception of sync paths configured with "restartRequired": false and created empty directories. These
are placed directly into the client's SPT installation.
Note
In the event of a download failure, ModSync will retry the download up to 5 times to prevent a failure causing the need for a full redownload.
Once all files have been downloaded, ModSync saves its updated PreviousSync.json and any removed files to ModSync_Data/RemovedFiles.json.
Using all of the information collected so far, and the files downloaded from the server, ModSync launches the ModSync Updater. This is a helper program shipped with ModSync that simply waits until Tarkov has closed, moves updated files into the correct locations, and deletes any removed files from the client.
Note
The ModSync Updater writes logs out to ModSync_Data/ModSync.log, so if you get any errors when the Updater is running, this is the place to check!
Several features of ModSync will let you as an admin or user modify the mod's behavior. The sections below break down some of the common use cases and how operation of the mod changes with them active.
Rescue mode is activated when the ModSync client is incompatible with the current server version. In rescue mode, when the client requests the server's list of sync paths and file hashes, the server will substitute the actual values with placeholders designed to force an update to the newer version of ModSync.
Typically this means forcing the client to download an updated BepInEx/plugins/Corter-ModSync.dll and ModSync.Updater.exe.
Enforced sync paths modify behavior in a couple of key ways to help ensure client installations are as similar to the server as possible. These modifications in behavior work together to achieve that goal.
When working with enforced paths, local exclusions are ignored entirely.
The comparison used to determine updated files is actually much simpler when enforced is true. The server's hash for a given file is simply compared to the client's. If the two don't match, then the file is marked as updated and queued for download.
Similar to updated files, when dealing with enforced paths the comparison for removed files is simplified. Removed files are simply any files that exist on client, but not on the server. In addition, the client configuration to disable file deletion is ignored entirely for files in an enforced path.
Warning
This can be a bit of a footgun. Admins must be careful to add all files the client will need or generate to their install or add them as server-side exclusions
When running with the FIKA dedicated plugin, ModSync essentially sets all sync paths to use the silent behavior. Update prompts are skipped and as soon as files are downloaded, SPT will be stopped and updates applied.
In addition, a client-side exclusion file will be automatically created and populated with mod files disallowed by the FIKA dedicated plugin.