Keeping backup files

[X1353135]

Thank you for your improvement of JFileSync.
It would be nice if JFileSync3 supports a option to move files to be overwritten or deleted during the directory synchronization into a individual directory selected in sync profile as backup.
I prefer to keep many generations of backup files as history, then delete manually later.

[X1353135]

It would be nice if backup directory is organized like following because user can delete old history easily by manual operation.

top_folder_of_backup_selected_in_sync_profile
	|
	+ 20170601_010203 <--- synced date & time
	|	|
	|	+ folder_path_relative_to_sync_top
	|		|
	|		+ synced_file1
	|		+ synced_file2
	|		+ synced_file3
	|
	+ 20170602_040506 <--- synced date & time
		|
		+ folder_path_relative_to_sync_top
			|
			+ synced_file2
			+ synced_file3
			+ synced_file4

[mgoellnitz]

Thanks for your feedback. Please note that also the original JFileSync got some improvements after the fork, related to WebDAV backends. But of course it still lacks the support for encryption in different ways and the compression alongside.

Since JFileSync3's main goal is to synchronize I don't see a compatible way to add your suggestions while leaving the plain syncing (with optionally several using systems involved per folder) intact. So your idea might be a little bit too far away from the original focus of JFileSync(3).

The problem stems from the date which needs to be ignored on comparison and storing back files to the original location. This resides quite at the core of JFileSync(3).

While I'm using JFileSync3 on a daily basis as one component for my backups it's focus definitely was not in keeping a file's history.

Do you have any idea on how to incorporate this extension in a compatible way? Perhaps with the option to tell how many versions of a file should be kept. Otherwise it might be a good idea to evaluate other options for you scenario provided that the add all the other features you might need like encryption, online backends, and compression.

Looking forward to hear your ideas.

[X1353135]

Thank you for your comment.
Are you assuming that files are backed up under the folder to be synced?
I think it's OK to forbid to select a folder under sync target directory as backup destination.

I'm assuming these folder structure.

Case 1:

file_system
	|
	+folder_to_sync
	|	|
	|	+file1
	|	+file2
	|	+file3
	|	+file4
	|
	+folder_for_backup
		|
		+ 20170601_010203 <--- synced date & time
		|	|
		|	+ folder_path_relative_to_sync_top
		|		|
		|		+ file1
		|		+ file2
		|		+ file3
		|
		+ 20170602_040506 <--- synced date & time
			|
			+ folder_path_relative_to_sync_top
				|
				+ file2
				+ file3
				+ file4

Case 2:

file_system_to_sync
	|
	+folder
		|
		+file1
		+file2
		+file3
		+file4
	
file_system_for_backup
		|
		+ 20170601_010203 <--- synced date & time
		|	|
		|	+ folder_path_relative_to_sync_top
		|		|
		|		+ file1
		|		+ file2
		|		+ file3
		|
		+ 20170602_040506 <--- synced date & time
			|
			+ folder_path_relative_to_sync_top
				|
				+ file2
				+ file3
				+ file4

I'm sorry, I'm not very good reader of English, Perhaps I may be misunderstanding your comment.
Could you please tell me if I miss the point what you are afraid of?

[mgoellnitz]

My point is not so much the backup, but the recovery side of the process - and how to discover the correct files and their versions.

JFileSync(3) tries to keep folders in sync (while still I'm using this as a PART of my Backup scenario). In your scenario file_system_to_sync is not in sync with file_system_for_backup. So the easiest answer is "you're trying to use the wrong tool", since keeping in sync ignores anything like a file history and versions.

On the other hand, this still might be a valid scenario taken into account what I'm doing with JFileSync3 on a daily basis. I simplified this to one file system:

current_scenario
	|
	+folder_to_sync_A
	|	|
	|	+file1
	|	+file2
	|	+file3
	|	+file4
	|
	+folder_to_sync_B
	|	|
	|	+file1
	|	+file2
	|	+file3
	|	+file4
	|
	+folder_for_backup
	 	|
	 	+file1
	 	+file2
	 	+file3
	 	+file4

When I sync folder_to_sync_A all the latest files and versions got to folder_for_backup. When I then sync folder_to_sync_B I get the latest files and versions from folder_for_backup and can store my latest stuff in there - simple as that. So all three base folders are in sync (In real world scenarios folders A and B reside on different machines and folder_for_backup is some where in the cloud).

Lets take your file system layout:

my_question
	|
	+folder_to_sync_A
	|	|
	|	+file1
	|	+file2
	|	+file3
	|	+file4
	|
	+folder_to_sync_B
	|	|
	|	+file1
	|	+file2
	|	+file3
	|	+file4
	|
	+folder_for_backup
		|
		+ 20170601_010203 <--- synced date & time
		|	|
		|	+ folder_path_relative_to_sync_top
		|		|
		|		+ file1
		|		+ file2
		|		+ file3
		|
		+ 20170602_040506 <--- synced date & time
			|
			+ folder_path_relative_to_sync_top
				|
				+ file2
				+ file3
				+ file4

Folders folder_to_sync_A and folder_to_sync_B are not in sync with folder_for_backup. When syncing with folder_for_backup, how would you expect JFileSync3 to behave to store files (and which version?) back from folder_for_backup to folder_to_sync_A or folder_to_sync_B?

This is a main difference between multi-version backups and single versions syncs with multiple storage instances (which is what I feel to need to keep my files safe, but only the latest version).

Does that shed any light on the problem as I see it. Any Ideas to resolve this? Or should be declare this out of scope for JFileSync(3).

One more point: File with versions. Ok, understood. But what about the svn/git approach take for source code repositories of keeping the right files in the right versions together in one iteration (as opposed to CVS)? This was not part of your question though...

[X1353135]

Thank you for you taking time to answer and I'm very sorry for late response.
Is the synchronization mentioned in your comment between folder_to_sync_A(B) and backup folder?
The synchronization in my scenario is between mirror1 and mirror2, and the backup folder is just a folder to keep histories of trashed files from a mirror.

before_sync
	|
	+morror1_folder_to_sync
	|	|
	|	+file1
	|	+file2
	|	+		<--- deleted
	|	+file4
	|
	+mirror2_folder_to_sync
	 	|
	 	+file1
	 	+file2
	 	+file3
	 	+file4	<--- updated


after_sync
	|
	+mirror1_folder_to_sync
	|	|
	|	+file1
	|	+file2
	|	+
	|	+file4	<--- updated
	|
	+mirror2_folder_to_sync
	|	|
	|	+file1
	|	+file2
	|	+		<--- deleted
	|	+file4
	|
	+backup_folder_of_mirror1
	|	|
	|	+ 20170601_010203
	|	 	|
	|	 	+ folder_path_relative_to_sync_top
	|	 		|
	|	 		+ file4	<--- original file before update
	|
	+backup_folder_of_mirror2
		|
		+ 20170601_010203
		 	|
		 	+ folder_path_relative_to_sync_top
		 		|
		 		+ file3	<--- original file before delete

I just want a feature to trash obsoleted files into a folder like recycle bin, instead of directory deleting, in order to give users a chance to salvage files deleted unexpectedly.

what about the svn/git approach take for source code repositories of keeping the right files in the right versions together in one iteration

Thank you for your comment, there is a long story about it.
I recently joined a software development team which have very conservative culture.
The team shares master source code just putting on windows shared folder, and doesn't keep any history(The team just zip the folder as a backup some time).
Thus, I sync folder to a mirror on my pc, then commit to git repository(also on my pc) per a day to keep modification history personally.

windows_shared_folder
	|
	+project1_master
		|
		+src	<--- sync

my_pc
	|
	+project1_mirror
		|
		+.git
		+src	<--- sync and commit to git

On a day, the team changed the location of master source code on windows shared folder.
Thus, I also changed(recreated) my sync configuration to follow it, but I made a mistake.
I synced project1_master to project1_mirror instead of syncing project1_master/src and project1_mirror/src.
As a result, .git folder was deleted and lost forever.
Of course, user is a responsible to configure sync tool correctly, but the user has a chance to restore files if the sync tool support to keep histories of trashed files as backup.

[clach04]

@mgoellnitz I concur with the thought that JFileSync3's focus is on syncing files, not backing up. I don't think adding backup/history support makes sense. However I think I have a possible answer to your question, "how to incorporate this extension in a compatible way?".

How about adding JSR 223 support (or some other scripting option) and then fire callbacks. That way if backups are something people are interested in, it can be developed outside of JFileSync3 and users could share scripts?

I personally like Python so I'd suggest https://github.com/scijava/scripting-jython, Kotlin is all the rage and might be a good choice. What ever is chosen would need to have decent file system and file IO support (e.g. last time I looked, LUA did not have a way to list the existing files in a directory out of box)

@X1353135 would that satisfy your needs?