(Sub) Folder Rename Causes Reuploading All Files...

[pjank]

Hi,

 

This was in my eyes one of the biggest advantages of BTSync over some other syncing solutions. There have been some problems from time to time, but I thought they were all fixed by now. Unfortunately, I've just experienced (twice!) an unwanted behaviour in what I would call a simple folder rename. I could swear this was working 1/2 years ago in identical situations, but now - seems broken. :/

 

I'm talking about detecting a folder rename. That's a folder inside a "standard Sync folder", not the sync folder itself. A folder containing hundreds/thousands of files.

 

What I would expect:

- Sync just *instantly* replicates the same folder rename operation on all other machines.

 

What actually happened:

- Sync on the originating machine actually does detect there was a rename ("History" shows all the events like "Renamed file <oldfolder>/<filename> to <newfolder>/<filename>"). That's half success :-O

- Sync on all the other machines unfortunately still treats this as a REMOVE + ADD operation, causing unnecessary TRANSFER of gigabytes of data, wasted time and space (moves the "old" folder to Archive). Very bad! :-(

 

Do I ask for too much? :-/

 

Configuration:

- source machine: Linux, others: Linux / Windows

- version on all: 2.2.5

 

 

BTW. If only the "Archive" contained all the files with the ORIGINAL timestamps - I could maybe just stop the btsync process, restore folder from the backup, rename it manually and then let btsync reindex it again (hopefully without transferring the whole files over the network). But it does not - all the archived files' modified time-stamps are updated to the date/time of this archiving event! WHY!?

 

 

[Helen]

pjank,

 

renaming actually *is* deleting+creating an item from the system't point of view. and while on the original peer it can be detected as renaming in History, on other peer there is nothing left but to obey the system's notification received - "removed" and "added", which you also see in History.

 

Sync performs this removing/adding via Archive, the old files (from old_name_folder) are indeed moved to archive, but later on they get restored from there, right for the purpose of *not* re-syncing. 

Which doesn't happen in your case, right? Instead you end up with two sets of the subfolder - older in Archive and newer (renamed) re-synced, right? 

 

Can you please rename some test subfolder/file and send the logs to the support . Please indicate which exactly file/subfolder is affected, to ease tracing it down in the log. Thanks!

[pjank]

the old files are indeed moved to archive, but later on they get restored from there, right for the purpose of *not* re-syncing

 

I didn't know that. Thank you for the clarification, Helen.

 

Which doesn't happen in your case, right? Instead you end up with two sets of the subfolder - older in Archive and newer (renamed) re-synced, right?

 

I did take a closer look and... that's actually not exactly true. It is kinda interesting though.

After few hours, when all the files have synced eventually, it looks like this:

1) Renamed folder A: 1221 files, ~400MB - the Archive on one target computer contains 998 of the same files (~340MB), and on another target machine - *exactly* the same 998 files as well.

2) Renamed folder B: 2644 files, ~1GB - the Archive (again, both target computers) contains exactly 1998 of the same files (~770MB).

In all cases that were the older files in both folders (first on the list when sorted by name or date) that were left in Archive, only the last (latest) 223 / 646 (folder A / B ) files were apparently restored and didn't have to be downloaded again.

 

The fact that both remote machines behaved in exactly the same way makes me think it's not random. There has to be some logic behind this   And that x998 number? That's suspicious....

 

Anyway, I will try to track it down with a debug log enabled next time I need to do similar rename. And until then... maybe those details above give you any hints, or raise new questions about what is going on?

 

Thanks,

PJ

[Helen]

PJ,

 

thanks for the details. The only reason for that behavior I can think of (without seeing logs) is that in some way those 223 / 646 files have same checksum as 998/1998 files . So when Sync was searching for a file with hash "XXXXXXXX" to restore- it found one and restored, the other one - had to be re-synced. But without logs I cannot account for all the 998/1998 files re-synced. 

 

So yes, please do send the logs to us. 

Or you can try to search it and see if you have "Trash: requested file was not found" in the log on target peers. Thanks! 

[pjank]

As it turns out, I've had logging enabled on one of the target peers :-o  Completely forgot about that

 

 

you can try to search it and see if you have "Trash: requested file was not found" in the log on target peers

 

And it is there - exactly 1998 times :-)

 

I've just sent the log file to the support.

[Helen]

in pjank's case re-syncing of the files happened due to reverse order of the file system notifications  arriving in the target peer - first was notification about file added, then - about file removal . Currently there is no quick fix for that, as there is little a peer can do to manage system notifications. Fixing that will perhaps require changing the logic of exchanging notifications, and this will require time. 

 

We will try to improve it in future releases.