overwritten by older version of file

[lpa]

BTSync overwrites newer files with older ones. This happens if files have been modified while one client is offline. When that client comes online BTSync overwrites files with that client's (old) versions. Why?

When I look at .SyncArchive on other clients I can see the deleted newer files with newer modification times than the old file, which overwrote them.

I have 4 machine setup:

1 always online, Linux (server);

2 mostly(just occasional restarts) online, one Linux, one Windows;

1 laptop, online when needed, Linux. (this is the client, that propagates old file versions)

All 3 Linux machines have same config, so I cannot blame laptop's config.

Yes, I have checked clocks on all machines.

And yes, permissions are fine, can create and modify files. (Anyway it would be lame to overwrite new files, because one client has old read-only copies)

I do not appear to be alone with this problem:

http://forum.bittorrent.com/topic/24146-sync-overwrites-new-files-if-a-node-is-away-for-a-while/

http://forum.bittorrent.com/topic/22382-new-files-overwritten/

http://forum.bittorrent.com/topic/24058-btsync-deleted-new-version-of-files-and-synced-the-old-one/

http://forum.bittorrent.com/topic/23272-issue-1170-replaces-new-files-with-older-ones-from-remote-server/

http://forum.bittorrent.com/topic/20104-reproducible-data-loss-same-as-old-file-overwriting-new-files/

 

http://forum.bittorrent.com/topic/25081-sync-to-computer-with-old-version-file/

http://forum.bittorrent.com/topic/19668-help-newer-files-overwritten-by-older-files/

[Harold Feit]

Case sensitivity filenames?

[lpa]

No.

Happens to all lowercase. Besides, other Linux hosts would behave same if it were case sensitivity issue.

[Harold Feit]

Windows doesn't have case sensitivity at a layer where it would actually prevent some of the problems caused by having two otherwise identically named files with different cases.

[lpa]

All due respect, I do not need a lesson on Windows vs. Linux filename limitations.

As I said earlier, the filenames are fine.

I will try to illustrate with an example.

Laptop was offline, I changed text file "test", turned on laptop it sent old version of "test" back to other nodes.

The outputs are from each machine.

Laptop before turning on btsync:

lpa@laptop % stat test   File: ‘test’  Size: 3         	Blocks: 8          IO Block: 4096   regular fileDevice: fe06h/65030d	Inode: 13635201    Links: 1Access: (0644/-rw-r--r--)  Uid: ( 1000/ lpa)   Gid: (  100/   users)Access: 2013-10-03 19:30:49.000000000 +0300Modify: 2013-10-03 19:30:49.000000000 +0300Change: 2013-10-03 19:31:59.143203957 +0300 Birth: -

Laptop after:

lpa@laptop % stat test  File: ‘test’  Size: 3         	Blocks: 8          IO Block: 4096   regular fileDevice: fe06h/65030d	Inode: 13635201    Links: 1Access: (0644/-rw-r--r--)  Uid: ( 1000/ lpa)   Gid: (  100/   users)Access: 2013-10-03 19:30:49.000000000 +0300Modify: 2013-10-03 19:30:49.000000000 +0300Change: 2013-10-03 19:31:59.143203957 +0300 Birth: -

Server before:

lpa@server % stat test  File: ‘test’  Size: 9         	Blocks: 8          IO Block: 4096   regular fileDevice: fe00h/65024d	Inode: 786920      Links: 1Access: (0644/-rw-r--r--)  Uid: ( 1000/ lpa)   Gid: (  100/   users)Access: 2013-11-04 00:08:26.000000000 +0200Modify: 2013-11-04 00:08:26.000000000 +0200Change: 2013-11-04 00:08:37.882082822 +0200 Birth: -

Server after:

lpa@server % stat test  File: ‘test’  Size: 3         	Blocks: 8          IO Block: 4096   regular fileDevice: fe00h/65024d	Inode: 786920      Links: 1Access: (0644/-rw-r--r--)  Uid: ( 1000/ lpa)   Gid: (  100/   users)Access: 2013-10-03 19:30:49.000000000 +0300Modify: 2013-10-03 19:30:49.000000000 +0300Change: 2013-11-04 00:21:04.481515684 +0200 Birth: -lpa@server % stat .SyncArchive/test  File: ‘.SyncArchive/test’  Size: 3         	Blocks: 8          IO Block: 4096   regular fileDevice: fe00h/65024d	Inode: 787922      Links: 1Access: (0644/-rw-r--r--)  Uid: ( 1000/ lpa)   Gid: (  100/   users)Access: 2013-11-04 00:08:37.000000000 +0200Modify: 2013-11-04 00:08:37.000000000 +0200Change: 2013-11-04 00:08:37.882082822 +0200 Birth: -

Desktop before:

lpa@desktop % stat test  File: ‘test’                                                      Size: 9         	Blocks: 8          IO Block: 4096   regular fileDevice: fe05h/65029d	Inode: 5505772     Links: 1Access: (0644/-rw-r--r--)  Uid: ( 1000/ lpa)   Gid: (  100/   users)Access: 2013-11-04 00:08:26.000000000 +0200Modify: 2013-11-04 00:08:26.000000000 +0200Change: 2013-11-04 00:08:38.392873583 +0200 Birth: -

Desktop after:

lpa@desktop % stat test  File: ‘test’  Size: 3         	Blocks: 8          IO Block: 4096   regular fileDevice: fe05h/65029d	Inode: 5505772     Links: 1Access: (0644/-rw-r--r--)  Uid: ( 1000/ lpa)   Gid: (  100/   users)Access: 2013-10-03 19:30:49.000000000 +0300Modify: 2013-10-03 19:30:49.000000000 +0300Change: 2013-11-04 00:21:06.153467994 +0200 Birth: -lpa@desktop % stat .SyncArchive/test  File: ‘.SyncArchive/test’  Size: 3         	Blocks: 8          IO Block: 4096   regular fileDevice: fe05h/65029d	Inode: 5505753     Links: 1Access: (0644/-rw-r--r--)  Uid: ( 1000/ lpa)   Gid: (  100/   users)Access: 2013-11-04 00:08:38.000000000 +0200Modify: 2013-11-04 00:08:38.000000000 +0200Change: 2013-11-04 00:08:38.392873583 +0200 Birth: -

Windows desktop before:

> dir2013.11.04.  00:08                 9 test

Windows desktop after:

> dir2013.10.03.  18:30                 3 test.SyncArchive> dir2013.11.04.  00:21                 9 test

Laptop obviously lacks file "test" in .SyncArchive since it ignored updated file and pushed its old one.

[jakurtz]

I'm having the same issue, as well. Three devices:

 

- Linux Mint 15 laptop

- Galaxy Nexus phone (with CyanogenMod installed)

- Windows 7 laptop

 

I'm not sure which device is propagating the older versions of the files, but your claim of the sometimes-online devices being the culprit sounds reasonable.

 

This needs to be fixed - or I can't use btsync. I lost a bunch of work the other day because of this issue, so now have started copying anything important to another folder - just in case.

 

I'm more than happy to provide further details if needed.

[athemelis]

I have the same issue as well - and it is also preventing me from "trusting" btsync with important files.

I use a number of devices, including Mac, Windows, Linux, Android - and not all of them are online at the same time.

I would like for when a device comes online and it sees a newer version of a file that was synced days ago - to not overwrite it with it's own older version.

[Harold Feit]

What specific version of Sync are you using?

[JimmyTheSaint]

Since this same problem was addressed at http://forum.bittorrent.com/topic/24856-problem-with-the-mobile-client/ I've had the older-replaces-newer situation happen a couple of times, but only when updating versions of the app on one or more devices. It's unsettling, but if you make sure to read the activity in the history tab, you can verify exactly what's happened, and things are recoverable. Once everything "settles down," I never get the reproducible error discussed in the earlier thread. It always seems to involve an Android client. By the way, my Linux clients sync 24/7, but a couple of my Windows machines go on- and offline frequently.

 

I've had another anomaly or two, always involving a client whose internet connection becomes unstable, but nothing I can document or reproduce intentionally.

[athemelis]

I'm running the following versions:

 

ubuntu saucy: 1.1.82

kubuntu saucy: 1.1.82

osx Mavericks: 1.1.82

windows 8: 1.1.82

Android Galaxy S4: 1.2.9

 

I see from the links above that perhaps I should upgrade the Android version to 1.2.12 - but although it says 1.2.12 in the Play Store, the version on my device says 1.2.9.

[JimmyTheSaint]

Stay current with the first post of the thread that's always first at the top of this forum.

 

EDIT: I should add that lately I'm seeing a new glitch. MyFile.txt is in sync, but there's also a file named MyFile.txt.SyncOld. How is this .SyncOld file supposed to be synced? I always end up deleting in manually.

[itcrowd]

I'm having the same issue and in my opinion the syncing is not working well. 

Based on the my sync.log the local file is older than remote file. It is very strange because I've modified the file on the local side only. 

• Windows: 7:  6.1.7601
• Desktop client version: 1.2.73
• Android version: 4.3 - Samsung Galaxy S4 (GT-i9505)
• Android client version: 1.2.9

Otherwise it can be reproduced anytime on desktop::

1., to be created a file with any contents on the Desktop... - the created file has attributes on the Desktop as follows:

Last write time: 2013.11.17 19:18:53,929

Creation time: 2013.11.17 19:18:53,928

Last access time: 2013.11.17 19:18:53,928

Change time: 2013.11.17 19:18:53,929

 

2., after this the text file is synced between Desktop and PC

3., to be modified the file on the Desktop again - the modified file has attributes on the Desktop as follows:

Last write time: 2013.11.17 19:22:28,722

Creation time: 2013.11.17 19:18:53,928

Last access time: 2013.11.17 19:18:53,928

Change time: 2013.11.17 19:22:28,722

 

4., after this the text file is synced and it seems to be that the file will be changed to before version - 2nd synced file has attributes on the Desktop as follows:

Last write time: 2013.11.17 19:18:53,000

Creation time:  2013.11.17 19:18:53,928

Last access time: 2013.11.17 19:18:53,928

Change time: 2013.11.17 19:23:45,270

 

The 'last-written' and 'change-time' attribute get changed together when someone writes to the file (or else when someone deliberately changes the last-written time).  

The 'change-time' is a file attribute but applies to all data streams. So, for example, if you change some property (e.g. read only) that will change the 'change-time' attribute only.

 

In my opinion in last case the file 'change-time' can be changed by Android sync, I don't know why, maybe it can be a bug and therefore Desktop client can see that the Android's file is newest.

 

It would be better, if others can check it and I hope it will be fixed as soon as possible.

 

Regards,

ITcrowd

 

 

[daishi4u]

I'm having the same problem as itcrowd

Sent from my SCH-S738C using Tapatalk

[vadimt]

This issue should be fixed in the latest version for android.

Please update your mobile devices to 1.2.12 build.

http://syncapp.bittorrent.com/android/BTSync-1.2.12.apk

Thanks!

[itcrowd]

Thanks, it is working well! When will it be on the market?

Tapatalk-kal küldve az én GT-I9505-el

[vadimt]

We are planning to turn on autoupdate soon.

Thanks!

[reggabegga]

I'm having the exact same problem as OP + multiple others across the forum. It appears the bug is not being looked into as it has been around for many months and versions.

I sincerly love BTsync and would like to trust it with all my data, but I'm cautious after experiencing this bug..

 

I know for certain that the issue is not Android related, as it can easily be reproduced without having the folder synced to the Android device.

 

I have 6 machines in my setup;

3 linux machines with version 1.2.73 - these are always on.

2 of them have encrypted (read only) secrets, 1 machine has read/write secret

1 windows machine with version 1.2.73 - almost always on. This machine has encrypted (read only) secret.

2 windows machines with version 1.2.73. These are my work and home computers and are turned on and off several times a day. Both have read/write secret.

 

How to reproduce;

1 Win machine with r/w secret (my work computer - computer A) edits a file, in this case my Keepass-file. The previous edit of the file was 4 days ago.

All machines except 1 Win with r/w secret (my home computer - computer B ) is online and the file is synced to the other 4 online nodes.

Computer A is turned off and after 2 hours in traffic I'm home and turn on computer B. I expect the newly edited file to be downloaded from the 4 online nodes, but instead, Computer B uploads its local file that was last edited 4 days ago, and replaces the new file on the 4 nodes with its 4 days old file.

 

Which log-files (from which nodes?) and what further details are needed for the devs to look further into this?

 

Thank you.

[vadimt]

Please turn on debug logging (http://forum.bittorrent.com/topic/12658-if-you-have-syncapp-issue/), reproduce this issue and send debug logs from your computer B and one of the other machine that has newer version of file to syncapp@bittorrent.com.

Many thanks!

[lpa]

I think I may have found the problem.

 

"storage_path" directory was not writable for user that runs btsync. And if it is empty, it tries to create directory ".sync" where binary is located, on Arch system it is /usr/bin. This directory, of course, is not writable for user.

Systemd journal indicated following problems:

SyncDb: failed to begin transaction to save metadata - 21Failed file save: /usr/bin/.sync//sync.dat.newShutdown. Saving config sync.datFailed file save: /usr/bin/.sync//settings.dat.new

So I changed storage_path value to "/home/user/.sync" and everything worked. Have not seen files overwritten with old ones since.

I must note, that all 3 of my linux nodes had this storage_path write problem, but since 2 of them where online most of the time, I did not experience any problems with wrong files.

 

What do I think about this?

1. BTSync node, that cannot write to "storage_path", should not push its old files to other nodes. Such behavior is fundamentally against the whole purpose of this software.

As I see it, such node, should either "just sit" and do nothing or not start at all. Giving clear error messages that it needs storage_path for functioning properly.

 

2. The two slashes in error log is confusing. At first I suspected, that the software tries an invalid path.

 

 

Can anyone, having same problem, confirm this?

Thanks!

[JimmyTheSaint]

This issue should be fixed in the latest version for android.

Please update your mobile devices to 1.2.12 build.

http://syncapp.bittorrent.com/android/BTSync-1.2.12.apk

Thanks!

I still intermittently have this old-overwrites-new problem with Android. It's only with my S4, which is my only Android device to change WiFi networks and 3G location (my Note 2 never leaves home WiFi). I can't see a specific pattern, but it seems to only arise when the S4 and/or the laptop that tethers to its data plan have a slow 3G network connection, I guess due to the random fluctuations of my phone company. Changed files still occasionally get clobbered, maybe one file every 5-7 days. But that's often enough to be very unsettling so that I'm now constantly checking history on different machines, which makes using btsync much more of a chore. No telling if I've lost stuff because I didn't track my history closely enough. Once, I had an emacs save-file (starts with a '#') that I simply could not delete from any other client: the S4 would always add it back. I deleted it from the S4, no problem. But the rest of the time I don't mess around with the file in question because it's got real data--I just fix things quickly. As far as I can tell, all content has always been recoverable from the archive folder. But that presumes noticing the problem in your history, which could easily be missed for a file that's rarely changed.

 

I currently suspect auto-sleep might have a connection (to this problem and to the losing sync issue), so I'll try going as long as I can without auto-sleep and see if the problem comes up. But sometimes I can go seven days or more without the problem, so there's no way to be conclusive.