Bitwise Posted February 19, 2015 Report Share Posted February 19, 2015 I just performed a sync using BitTorrent Sync v1.4.110. I am 100% certain BitTorrent Sync overwrote newer files with older files. I don't have any idea why it would do this, except for a serious bug. The file dates and file contents clearly show that new files were overwitten by old versions. I had performed a backup of some of the data before enabling the sync, and the backup confirms the problem. A little more detail: I had previously used BitTorrent Sync on these folders using 1.4.9x. Only 2 computers are using BTSync and they run the Windows 64-bit version. They are on the same wireless network. Relay and tracker servers are turned off for each sync folder. I hadn't run BTSync in over a month. I upgraded BTSync to v1.4.110 on both computers. As part of the BTSync upgrade on System #1 (which had many new files), I selected the option to run BTSync after the program version upgrade. Once it was upgraded and loaded, I ran BTSync on System #2 (which I had already upgraded to the latest version). It than began to sync. Everything looked fine and appeared to run smoothly. But when I checked the files, in many cases, old versions overwote the new versions. I now have to sift through thousands of files to determine the damage BTSync caused and try to rectify as much of the damage as possible. UPDATE: I just had a full-time computer programmer verify all the above facts, and he concluded there is a serious problem with BitTorrent Sync, and that it definitely overwrote NEWER files with OLDER ones in multiple cases. This seems to be a huge problem. Quote Link to comment Share on other sites More sharing options...
Digto Posted February 21, 2015 Report Share Posted February 21, 2015 (edited) I can confirm this problem.My setup: PC1 has newest file is read+write, PC2 is a linux server to act as cloud and readonly, PC3 has old file and is read+write. PC1 syncs to PC2 with the new file, PC3 is offline.PC3 goes online and syncs with PC2, now uploading the old file and overwriting the new file, PC1 is offline.PC1 goes online and syncs with PC2, now receiving the old file, PC3 is offline.The new file is now removed unless you have backup of it. The reason for PC1 and PC3 to both have write is because my laptop and desktop should both be able to add/update/delete files, but the newest file should ALWAYS overwrite the oldest one. Using 1.4.110 Beta on Windows 8.1,10 and Linux. Edited February 21, 2015 by Digto Quote Link to comment Share on other sites More sharing options...
Digto Posted March 24, 2015 Report Share Posted March 24, 2015 I am now running version 2.0.93 running on Windows 8.1, Windows 10 and Ubuntu. And this is still a problem, I fear that having more than one pc able to write in the folders is dangerous, so I'm going to try having only 1 pc that can write, and make the rest read only. Quote Link to comment Share on other sites More sharing options...
Stardust Posted April 10, 2015 Report Share Posted April 10, 2015 Same problem. I have been using btsync since December (documents and music) and lately a to sync folder with source code.Big mistake.Btsync has never really seemed to sync properly, and stops syncing frequently. Today, I needed to look at some source code on a fedora box and noticed that it hadn't synced for more than two weeks. So I restarted btsync, without it helping. BTsync still claimed everything was in sync, when it clearly wasn't. I disconnected and reconnected the share leaving the files in place. Big mistake. Every old file was copied from the stale folder to the remote fresh one overwriting two weeks of work. Luckily most of it was commited to a git repo, but still loads of work in progress was lost. BTW, you keep referring to version numbers, but neither the download nor the output when starting it mentions it. Anyway, I'm sure it is the latest, since the web UI didn't report an update was available.Also, do you really expect me to "turn on debugging" and recreate the problem? I mad enough as it is. Quote Link to comment Share on other sites More sharing options...
Stardust Posted April 12, 2015 Report Share Posted April 12, 2015 Same problem. I have been using btsync since December (documents and music) and lately a to sync folder with source code.Big mistake.Btsync has never really seemed to sync properly, and stops syncing frequently. Today, I needed to look at some source code on a fedora box and noticed that it hadn't synced for more than two weeks. So I restarted btsync, without it helping. BTsync still claimed everything was in sync, when it clearly wasn't. I disconnected and reconnected the share leaving the files in place. Big mistake. Every old file was copied from the stale folder to the remote fresh one overwriting two weeks of work. Luckily most of it was committed to a git repo, but still loads of work in progress was lost. BTW, you keep referring to version numbers, but neither the download nor the output when starting it mentions it. Anyway, I'm sure it is the latest, since the web UI didn't report an update was available.Also, do you really expect me to "turn on debugging" and recreate the problem? I mad enough as it is.Update. I was able to restore everything overwritten from the .sync/Archive folder, so there is a good thing.What I didn't mention i my initial bug report was that these were old 1.4 shares. I disconnected all clients, and recreated the shares as 2.0(?) shares, lets see how that works out. Also, the setup is 3 peers, Opensuse 13.1 and 13.2 on a LAN, and a Fedora 21 peer outside that (firewall/NAT). The fedora 21 peer was the one falling out of sync all the time. Quote Link to comment Share on other sites More sharing options...
mcv Posted April 30, 2015 Report Share Posted April 30, 2015 Can you check if all of your machines have synchronized system clocks?It happened to me a couple of times that new versions were replaced with old files. I found that at least one reason for that is desynchronized clock on one of the machines. It's easy and always reproducible. Since then I enabled automatic clock syncing from NTP on all of my systems.It's a pain that BTSync relies on system clock so heavily instead of doing some sort of own clock synchronization. Each instance could synchronize its own clock with the rest of BTSyncs, then calculate the difference between its clock and system clock and use such "corrected" time to see what files are really the new ones. Quote Link to comment Share on other sites More sharing options...
Stardust Posted April 30, 2015 Report Share Posted April 30, 2015 (edited) Can you check if all of your machines have synchronized system clocks?It happened to me a couple of times that new versions were replaced with old files. I found that at least one reason for that is desynchronized clock on one of the machines. It's easy and always reproducible. Since then I enabled automatic clock syncing from NTP on all of my systems.It's a pain that BTSync relies on system clock so heavily instead of doing some sort of own clock synchronization. Each instance could synchronize its own clock with the rest of BTSyncs, then calculate the difference between its clock and system clock and use such "corrected" time to see what files are really the new ones. All machines are running NTP. It would have to be a serious clock drift problem if BTSync suddenly thinks that two week old files suddenly are never than files timestamped minutes ago. Just to make sure, I rechecked all my clocks: ssh host1 'date +"%F %T.%N"'; ssh host2 'date +"%F %T.%N"'; ssh host3 'date +"%F %T.%N"'2015-04-30 13:15:37.3458029642015-04-30 13:15:37.4863563242015-04-30 13:15:37.489242791 Edited April 30, 2015 by Stardust Quote Link to comment Share on other sites More sharing options...
Brad@IO Posted October 21, 2015 Report Share Posted October 21, 2015 Any news on this?I've just discovered this massive bug and it's easily reproducable.It's a catastrophic scenario.New peers added with older versions of files simply overwrite all newer instances of those files on all peers!Wow.It's got nothing to do with clock syncing.Our tests have shown 10 year, 1 year, 1 month, 1 day, 3 hour old files overwriting files that are less than 10 min old.There's no hope of using BT sync in any kind of professional environment with this sort of issue. Can we get a formal response about this issue from support? Quote Link to comment Share on other sites More sharing options...
GreatMarko Posted October 21, 2015 Report Share Posted October 21, 2015 I've just discovered this massive bug and it's easily reproducable.Firstly, are you using the most recent version of Sync (which at time of writing is 2.2.5)? If not, please update to 2.2.5 to see if this solves your issue (for instance, 2.2.0 contained a fix for "Newer file replaced by old one from offline peer")If updating all your peers to 2.2.5 does not resolve your issue, please share your "easily reproducible" steps Quote Link to comment Share on other sites More sharing options...
RomanZ Posted October 22, 2015 Report Share Posted October 22, 2015 @Brad@IOAre you using advanced or standard folders? Quote Link to comment Share on other sites More sharing options...
Brad@IO Posted October 26, 2015 Report Share Posted October 26, 2015 @great marko: The windows peers are running 2.2.5, but is there a simple method for updating the version running on the synlogy NAS? I haven't been able to work out how to update that "package" so far from 2.0.93.@romanz: standard folders AFAIK Quote Link to comment Share on other sites More sharing options...
RomanZ Posted October 26, 2015 Report Share Posted October 26, 2015 @Brad@IOCould you please check it? It's important as issue you describe was fixed for Standard folders in 2.2.5 (while still exists for Advanced ones - its a bit more complex to fix it for advanced folders). Just open your folder share dialog, if it shows you "key" tab - it is standard one, otherwise it is Advanced. Quote Link to comment Share on other sites More sharing options...
Brad@IO Posted October 26, 2015 Report Share Posted October 26, 2015 @romanZ: Interesting. We've always picked STANDARD when we add a share, but it seems they've all been set up as ADVANCED folders. Maybe because the share window next presents a permissions option before you share the link. Am I right in thinking that if we see this permission option then we're setting up an ADVANCED folder? @great marko: found the manual instructions for updating BTsync on the Synology NAS. Done. Happy to test again, but will that only be useful if the folers are converted-to or re-shared as STANDARD? Quote Link to comment Share on other sites More sharing options...
Helen Posted October 27, 2015 Report Share Posted October 27, 2015 Telling Advanced and Standard folder apart is easy by just hovering mouse over it.No, no more testing needed, at least so far, thank you. We just wanted to confirm that this happens to Advanced folders, not Standard. Quote Link to comment Share on other sites More sharing options...
jacoch Posted January 15, 2016 Report Share Posted January 15, 2016 I'm facing the same problem. I had for several weeks only two computers synced together (Win10 and OSX). No trouble. Then I added a Synology NAS to the peers (running latest 2.2.5-1 version). That was the beginning of the troubles. By looking at the logs and file creation times, it seems that the creation time is wrong on Synology. It's set to the date the file was copied to. So when my computer changes a file, keeping its original creation time, the NAS thinks its version is newer and overwrite the changes. Why is file creation time used to check which file is newer? Most of the time, creation date is not changed by OSes. Is this problem limited to Synology? Quote Link to comment Share on other sites More sharing options...
RomanZ Posted January 16, 2016 Report Share Posted January 16, 2016 @jacochSync doesn't use the file creation time, but it is using file modification time instead. Also, when Sync delivers file, it adjusts its mtime so it will be the same as on other peers. I suspect that your Syno does not allow to alter mtime (technically, any FS on Linux can be mounted in the way it does not allow to change mtime). There is a simple test you can do to check it. SSH to your NAS as admin, go to some folder (which stays on the same FS as your synced folder) and try to "touch" files. See if it updates the mtime of a file. Quote Link to comment Share on other sites More sharing options...
jacoch Posted January 16, 2016 Report Share Posted January 16, 2016 (edited) Modifications times are fine. That's why I thought about the creation time. It's the only difference. So I looked at the logs more closely and noticed an error when setting attributes. That could be the cause of the troubles. Here are some interesting entries from my logs (it's a file uploaded on January 13th reverting to a version from January 6th). Maybe you can explain more clearly why the old file finishes to be sent to other peers. [20160113 14:56:25.726] MC[C17F] [7E7A]: will request nodes for /My/Path/To/File.pdf[20160113 14:56:27.187] MC[C17F] [7AF7]: will request nodes for /My/Path/To/File.pdf[20160113 14:56:27.308] MC[C17F] [7AF7]: processing nodes message for /My/Path/To/File.pdf[20160113 14:56:27.308] MC[C17F] [7AF7]: will request files for /My/Path/To/File.pdf[20160113 14:56:27.429] MC[C17F] [7AF7]: Local file My/Path/To/File.pdf is older than remote t:1452691269/1452693379 ot:28110/2210841 o:10CA626E2DA8A6F6AF702A6EE757641ABC7FEE40/10C0FE7D53478F7664D731BDBAEE10705B527E7A Wed, 13 Jan 2016 13:21:09 GMT / Wed, 13 Jan 2016 13:56:19 GMT[20160113 14:56:27.429] JOURNAL[C17F]: got file from remote: "My/Path/To/File.pdf" state: 1 type: file total:17 have:17 size:540230 t:1452693379 mt:1452693379 ot:2210841 o:10C0FE7D53478F7664D731BDBAEE10705B527E7A h:813725282900FCC03403E890737182DAB70E0A2D Wed, 13 Jan 2016 13:56:19 GMT[20160113 14:56:27.453] Update have pieces for file "/volume1/Plurial/Internet 08/My/Path/To/File.pdf", was: 17, now: 17[20160113 14:56:27.453] SyncFileEntry: writing file attributes to file "/volume1/Plurial/Internet 08/My/Path/To/File.pdf", mt:1452693379 Wed, 13 Jan 2016 13:56:19 GMT[20160113 14:56:27.453] SyncFileEntry: Failed to write attributes for file /volume1/Plurial/Internet 08/My/Path/To/File.pdf - 1[20160113 14:57:33.850] Update have pieces for file "/volume1/Plurial/Internet 08/My/Path/To/File.pdf", was: 17, now: 0[20160113 14:57:33.875] Update have pieces for file "/volume1/Plurial/Internet 08/My/Path/To/File.pdf", was: 0, now: 17[20160113 14:57:33.875] JOURNAL[C17F]: entry "/volume1/Plurial/Internet 08/My/Path/To/File.pdf" state: size:540230 mt:4294967295 t:1452693453 h:813725282900FCC03403E890737182DAB70E0A2D (new torrent for file) Wed, 13 Jan 2016 13:57:33 GMT[20160113 14:57:33.876] JOURNAL[C17F]: setting time for file "/volume1/Plurial/Internet 08/My/Path/To/File.pdf" to 1452693453 Wed, 13 Jan 2016 13:57:33 GMT[20160113 14:57:33.876] SyncFileEntry: Set owner time for entry "/volume1/Plurial/Internet 08/My/Path/To/File.pdf" to 28266[20160113 14:57:33.877] JOURNAL[C17F]: entry "/volume1/Plurial/Internet 08/My/Path/To/File.pdf" state: size:540230 mt:1452084885 t:1452693453 h:813725282900FCC03403E890737182DAB70E0A2D (file updated) Wed, 06 Jan 2016 12:54:45 GMT / Wed, 13 Jan 2016 13:57:33 GMT[20160113 14:57:41.888] MC[C17F] [7AF7]: will request nodes for /My/Path/To/File.pdf[20160113 14:57:42.008] MC[C17F] [7AF7]: processing nodes message for /My/Path/To/File.pdf[20160113 14:57:42.008] MC[C17F] [7AF7]: will request files for /My/Path/To/File.pdf[20160113 14:57:42.097] MC[C17F] [7E7A]: will request nodes for /My/Path/To/File.pdf[20160113 14:57:42.130] MC[C17F] [7AF7]: Local file My/Path/To/File.pdf is newer than remote t:1452693453/1452693379 ot:28266/2210841 o:10CA626E2DA8A6F6AF702A6EE757641ABC7FEE40/10C0FE7D53478F7664D731BDBAEE10705B527E7A Wed, 13 Jan 2016 13:57:33 GMT / Wed, 13 Jan 2016 13:56:19 GMT[20160113 14:57:42.130] MC[C17F] [7AF7]: will send file /My/Path/To/File.pdf Edited January 16, 2016 by jacoch Quote Link to comment Share on other sites More sharing options...
RomanZ Posted January 18, 2016 Report Share Posted January 18, 2016 @jacochStill it looks like the file system mounted does not allow to set modification time. Although, it usually happens when one mounts FS manually. As I understand, you did not perform any manual operations on your FS, it was partitioned and mounted by Syno DSM, right? If yes - could you please let me know which DSM version you use and which FS did you choose? Thanks. Quote Link to comment Share on other sites More sharing options...
jacoch Posted January 18, 2016 Report Share Posted January 18, 2016 Well, I would not say that it doesn't allow to set file modification time. It works properly on the shares. But maybe BTSync client doesn't support my configuration yet. I'm using DSM 6 beta and btrfs as FS. Quote Link to comment Share on other sites More sharing options...
Brad@IO Posted January 23, 2017 Report Share Posted January 23, 2017 Hi Guys, bumping this old thread to see if there's new info. When I had problem with older files overwriting newer ones back in Oct 2015, RomanZ, confirmed that the issue <snip>...was fixed for Standard folders in 2.2.5 (while still exists for Advanced ones - its a bit more complex to fix it for advanced folders). As a result we've never used Advanced folders, but we would like to be able to take advantage of them. I've scanned the change logs several times since, but been unable to find any mention of a specific fix. Is this issue fixed for Advanced folders now? cheers Quote Link to comment Share on other sites More sharing options...
Brad@IO Posted January 24, 2017 Report Share Posted January 24, 2017 It would be good to have a clear clarification, but I guess for now we should assume that the problem still exists...? Quote Link to comment Share on other sites More sharing options...
Helen Posted January 25, 2017 Report Share Posted January 25, 2017 The cause, discovered then, in that reported case, is fixed. Quote Link to comment Share on other sites More sharing options...
Brad@IO Posted January 25, 2017 Report Share Posted January 25, 2017 That response doesn't clarify where we are at regarding older files overwriting newer files in Advanced folders. There's an increasing number of companies we deal with where discussion of Resilio sync comes up as an option and we always mention the problem with older files overwriting newer ones. It would be great to be able to know that issue has been 100% fixed, with no caveats. Are you saying: the issue with standard folders discussed in the thread is fixed but the specific known issue with Advanced Folders referred to by RomanZ still exists or the specific known issue with Advanced Folders referred to by RomanZ is fixed but overwriting of new files with old still occurs with Advanced Folders due to other known issues there are NO known, reported or outstanding issues regarding overwriting of new files with old in Advanced folders. Is the answer 1, 2 or 3? Or, if it's easier: As previously stated, we just want to know: Can we use Advanced Folders without the risk of older files overwriting newer files. Is the answer Yes or No? I can start a new thread if it will lead to a clear YES or NO answer. Quote Link to comment Share on other sites More sharing options...
Helen Posted January 27, 2017 Report Share Posted January 27, 2017 it's 3: the issue reported and discussed here, referred to by Roman is fixed. There are a lot of other use cases when older file may get synced - for example, it's being edited at the same time on different computers by different people. It's out of Sync's control and whoever saves and closes the file last, uploads its copy to others, even though those others may have their "latest" copy saved some 10 seconds before. and alike. These cases do not depend on Standard/Advanced So I cannot tell you for sure "Yes or No", I don't know all your use cases and I cannot predict them. When it comes to this particular issue reported, discussed and referred to here by Roman- Yes, you can. Quote Link to comment Share on other sites More sharing options...
someuser Posted March 23, 2018 Report Share Posted March 23, 2018 I can confirm that this STILL happens as of version 2.5.12 (1191) I had a previously synced machine offline for a month. Fired it up, once it completed the sync. All of the old outdated files overwrote the newer files on the machine that's always online. This event has convinced me to uninstall this unreliable sync service. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.