Files Being Rolled Back

[mosteo]

I'm having this problem and don't know how to go about diagnosing it. Hope someone can give some pointers.

 

I have 5 computers, all of them Ubuntu (one 15.04, the rest 14.04.02), sharing a quite large folder (~50G size, ~150000 files). Clocks are properly synchronized now, they weren't in the past (with differences under 2 minutes) but after fixing that the issue still happens (and besides I think small clock differences aren't critical). Of these computers, only one is a laptop being sometimes off, the rest are always running.

 

Btsync is from the unofficial server repo:

$ wajig policy btsync
btsync:
  Installed: 2.0.0-2

 

The symptoms are of two kinds: after some time, a deleted folder tree reappears but without files. This, per se, is a nuissance but not that serious. However, I've recently noticed that changes in files are being rolled back. I see this clearly in some mercurial repos within the folder. After making sure the repos .hg folder is up-to-date with an off-site upstream repository, my local files show changes that are exactly their state before the last commit. I.e., their last changes have been lost.

 

I've enabled debug logs. One question would be, will these logs tell me from which computer originates the rollback for a particular file? Also, any suggestion for a methodic diagnostic? Every time I've tried to trigger this behavior manually I've miserably failed.

 

Thank you!

[tomzooi]

I have the same problem, and it appears btsync is doing some weird stuff, it likes updating files that were read only, consider the section of my log files containing some weird stuff about an mp3 file (please don't judge my music taste ) that was only read (as in: played).

http://pastebin.com/TF0Wpakv

 

Apart from that, my files keep being rolled back, which is very annoying if you casually coding something, save and find everything working, come back later and everything you just did is gone and not working any more....

 

I never had these problems, since then these things changed:

- updated to 2.0.128

- installed android app and connected it

- changed /etc/fstab rules to make my user of my files, both sync folders are on ntfs partition, mounted with same options:

 

/dev/sda5 /storage/2 ntfs-3g defaults,locale=en_US.utf8,uid=tom,gid=users 0 0

 

This issue is of course not workable, every time I start sync I run the risk ruining whatever I just made...
 

[mosteo]

@tomzooi, the diagnose I got from support after checking my logs was that some process was touching continuously my files in one computer. Thus, they were always being synced from that computer as if they were newer than the ones I was actually editing in another place.

 

I don't know how that could have happened, and never managed to find the cause. I was recommended to use fatrace or similar to try to pinpoint the culprit, may be that will help you. In my case, I've fallen back to only use btsync for non-important inmutable things like installable files, music and photos.

[tomzooi]

I haven't found a process that touches my files on the other device, currently there's even no processes running that are touching them (because I stopped btsync), normally there's only stuff like filebrowser and btsync listed after lsof.

 

I am actually guessing that through some weird quirk btsync itself is been touching the files and interpreting that as an update. Or the programs I run that only read the files are registering "touches" somehow, but I do not know how to check that

Otherwise I can't make any sense out of it since the modification times on the file are correct and clocks in sync.

 

I defaulted back to rsync btw, on demand syncing without interface, but using sftp mount fast and secure .

[kodyb]

This is happening for us too. The rogue machine that is deleting files is a Windows 7 machine running Sync 2.0.128 (36), which is the same as the rest of the clients on the network.

We've disabled the client on that machine for now, and the rest of the machines are still working fine.

 

The machine was running a virus scan at the time. Can this impact the sync capability?

[tomzooi]

Virus scan should not write to files, hence not change change time (ctime) Which is what btsync relies on according to earlier messages on this board.

[Helen]

@all,

 

Yes, Sync relies on ctime, which may be different from mtime. If Sync is off and file is changed/touched/moved/etc, ctime will equal the time Sync will start, and this file will be synced, regardless of what mtime it and other peers have. 

So checking what might be updating the file - is the first thing in debug. 

 

Also, workflow is a key factor there, cause there may be a race between when changes are saved, then file opened again, re-synced, closed, detected, etc. We had multiple cases when same file was edited at the same time on different machines, and the last detected ctime update got synced. 

 

So every person is invited to send the logs for analysis, with as precise description of the workflow as possible. Thank you. 

[tomzooi]

@all,

 

Yes, Sync relies on ctime, which may be different from mtime. If Sync is off and file is changed/touched/moved/etc, ctime will equal the time Sync will start, and this file will be synced, regardless of what mtime it and other peers have. 

So checking what might be updating the file - is the first thing in debug. 

 

Also, workflow is a key factor there, cause there may be a race between when changes are saved, then file opened again, re-synced, closed, detected, etc. We had multiple cases when same file was edited at the same time on different machines, and the last detected ctime update got synced. 

 

So every person is invited to send the logs for analysis, with as precise description of the workflow as possible. Thank you. 

I think I understand this partially, can you explain the first part a bit clearer, sync changes the ctime of files that are changed before it has started (and before it has finished indexing?) to the start time of sync?

 

Secondly, this race, why is the time files are opened, closed and detected important? Can you give an example of this race (going wrong)?

 

As soon as I get the chance (likely somewhere next week), I will give sync another try and keep a close watch on the logs. I would also like to see the logs of an Android device I would like to connect (though I ll start with 2 devices first) but I cannot find them in the place they are supposed to be with my file explorer (sidenote: the device is not rooted), the only way I can share it with you is through get help option (without letting me see it myself?), or is there an option to get it and post it here together with the relevant sections of the other three logs?

[kodyb]

@all,

 

Yes, Sync relies on ctime, which may be different from mtime. If Sync is off and file is changed/touched/moved/etc, ctime will equal the time Sync will start, and this file will be synced, regardless of what mtime it and other peers have. 

So checking what might be updating the file - is the first thing in debug. 

 

Also, workflow is a key factor there, cause there may be a race between when changes are saved, then file opened again, re-synced, closed, detected, etc. We had multiple cases when same file was edited at the same time on different machines, and the last detected ctime update got synced. 

 

So every person is invited to send the logs for analysis, with as precise description of the workflow as possible. Thank you. 

 

Where do we send the logs to?

 

I've got an issue where one rogue computer continues to choose to delete files, despite a full reinstallation of BT Sync.

 

-K

[kodyb]

Do I simply submit a section of the sync.log to pastebin?

http://pastebin.com/vQWP3wid

 

 

Where do we send the logs to?

 

I've got an issue where one rogue computer continues to choose to delete files, despite a full reinstallation of BT Sync.

 

-K

[mosteo]

Just to add a data point, in my case I was working only on one computer; the one identified as rolling back files was supposedly idle.

[Helen]

I think I understand this partially, can you explain the first part a bit clearer, sync changes the ctime of files that are changed before it has started (and before it has finished indexing?) to the start time of sync?

When Sync is off, it has no chance to determine when exactly file was changed. It will determine that only when it's turned on. ctime - is not the file's attribute, it's the temstamp in Sync's database = time when Sync detected file change. 

 

 

Secondly, this race, why is the time files are opened, closed and detected important? Can you give an example of this race (going wrong)?

1) person1 updates file on peer1. closes the file.

2) same moment person2 opens same file on peer2, works on it.

3) Sync1 detects file change.

4) person 2 closes the file - so file2 is the newer for both people.

5) Sync1 starts uploading the file. 

6) Sync2 it starts syncing file1. file2 is overwritten. 

then it depends on the app where the file is edited, if on step 4) that app closed and updated the file and notified system about it, if it will inform person2 on step 6) that the file has changed while it's still open, and manymany other use cases. That is why description of the workflow is much appreciated. 

Sync just syncs the bits of files that it's notified about. 

 

Instruction for log , overall information on how to report an issue 

[tomzooi]

When Sync is off, it has no chance to determine when exactly file was changed. It will determine that only when it's turned on. ctime - is not the file's attribute, it's the temstamp in Sync's database = time when Sync detected file change. 

 

Instruction for log , overall information on how to report an issue 

Does this off-time include the indexing by sync or is it considered "on" as soon as it is started?

 

Furthermore, the instruction for log do not say a single thing about Android, and how to get debug logs from there, so considering that my (optional) third device is my Android Smartphone, which actually might be the culprit is a bit useless, I like to look into log files myself, but I haven't been able to find them yet.

[Helen]

Does this off-time include the indexing by sync or is it considered "on" as soon as it is started?

depends

 

 

 

Furthermore, the instruction for log do not say a single thing about Android, and how to get debug logs from there, 

The instruction for logs has bold point 5. Mobile devices

In order to send logs from a mobile device, send a new feedback and agree to attach a log. In this feedback indicate where the log refers to (for example, the support ticket the logs were requested at) or describe the problem in detail.

[JimmyZ]

I'm having similar issues, I have an always on Debian server, two Windows desktop, one Windows laptop, all running 2.0.128

 

the file being rolled back comes from the laptop

 

I always shutdown desktops when I stop using them, but with the laptop I just close the lid so it sleeps. I think it might be related.

 

BTW: the history interface is a major PITA to use, at least show which directory does the entry belong to, option to show absolute time stamps, and a search/filter.

 

by googling I found similar issues was fired two years ago, I'm losing faith in this product, switching back to git.

Edited July 14, 2015 by JimmyZ