hhz

Bug Report: btsync Upgrade can cause removed files to reappear

Recommended Posts

Ok, here's what happened.

Four machines with btsync version $old. All synced.

Now upgrade three of them to btsync version $new. This has a new index file format and requires the machine to re-index the shared files.

Let $new re-index on the three upgraded machines, but let the fourth machine stay offline.

Then remove a few files, create a few other files and let the three $new machines sync each other.

Finally, boot the fourth machine, upgrade its $old version to $new. It will now re-index its content and re-transmit the old files, which had been removed on the other three machines.

Tada! The removed files are back on all four machines.

Share this post


Link to post
Share on other sites

This is as designed. Keep in mind that upgrading due to a changes in protocol and indexing is similar to a new Sync version installation. So your case is I have Sync on three machines and deleted a file, then install Sync on a new machine where this file exists. So it will be treated as someone recovered the file.

It might be not exactly how you expect it to work, but this was a very though upgrade for us.

HTH

kos

Share this post


Link to post
Share on other sites

But that defeats the original purpose of btsync to have it run on many machines that independently sync each other.

If an admin has to make sure that all users of a shared directory must upgrade at the same time and when all machines are in sync, it's not decentralized anymore.

Sorry, I still think that using timestamp and file existence to find out what to sync is the wrong approach. I have been using btsync for a very short time now and already stumbled over sync issues because of this several times.

Share this post


Link to post
Share on other sites

I presume that an actual change history that can be merged from several distributed machines (a la git-annex or Sparkleshare) would be more robust against the kind of sync issues I keep having with btsync right now.

Share this post


Link to post
Share on other sites

But that defeats the original purpose of btsync to have it run on many machines that independently sync each other.

Not really, the issue here is that 1.1.27 was a big update and wasn't backward compatible with previous versions. Due to incompatibility it wasn't really an upgrade. We stabilizing Sync and I hope there will be no more such incompatibility cases.

Sorry, I still think that using timestamp and file existence to find out what to sync is the wrong approach. I have been using btsync for a very short time now and already stumbled over sync issues because of this several times.

We know how to do it right, but first step we had to make is versioning. So stay tuned.

Share this post


Link to post
Share on other sites

This is as designed.

Then your design is flawed. An upgrade (regardless of how big the changes are) should never lead to data loss.

I (kinda) get the decision to use the simplest conflict resolution method there is - last added wins -, but that only works for changed files after the first initial sync has been done. For all other cases the user should be asked what to do (e.g. when adding a new folder with an existing secret there should be a option to decide whether to replace local or remote files on conflict), and like I said, a "simple" (for the user) update should not change the status quo.

Share this post


Link to post
Share on other sites

I presume that an actual change history that can be merged from several distributed machines (a la git-annex or Sparkleshare) would be more robust against the kind of sync issues I keep having with btsync right now.

I agree with you that this is right way to go.

Then your design is flawed. An upgrade (regardless of how big the changes are) should never lead to data loss.

This is definitely a pain and not something we intentionally put to the system, however you have to understand that root cause of the issue - you upgrade machines in the middle of sync. I.e. two upgraded machines perform some changes and then you bring third machine that cannot be aware of these changes due to an incompatible protocol.

Share this post


Link to post
Share on other sites

ChrisH, please note that I did not experience data loss. I only had files restored that should have been removed.

koz, I did not upgrade in the middle of a sync. When I turned off the fourth machine, its sync had been completed. When I turned it on, the three others were synced, as well, but in a newer state.

Still, the result was unexpected.

Share this post


Link to post
Share on other sites

ChrisH, please note that I did not experience data loss. I only had files restored that should have been removed.

Okay, I had changed files overwritten with their old versions.

koz, I did not upgrade in the middle of a sync. When I turned off the fourth machine, its sync had been completed. When I turned it on, the three others were synced, as well, but in a newer state.

Exactly.

Share this post


Link to post
Share on other sites

however you have to understand that root cause of the issue - you upgrade machines in the middle of sync. I.e. two upgraded machines perform some changes and then you bring third machine that cannot be aware of these changes due to an incompatible protocol.

I don't think that's the root cause, but okay. Like someone else already said in this thread, if you have to centrally synchronise and manage updates on all clients then a decentralized sync solution kinda loses its value.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.