umputun Posted April 24, 2013 Report Share Posted April 24, 2013 Hi,I have some strange behavior and to me it looks like a bug. I have a folder shared with read-only secret across ~100 users. At the same time it shared with full-access secret for some limited number of "admins". Each admin can add new file - this part works just fine. However it seems to be impossible to remove a file - even for admins it acts like "read-only", i.e. removed file get restored automatically.Is this expected behavior or unexpected bug? Any idea how file can be deleted, maybe some workaround? Quote Link to comment Share on other sites More sharing options...
umputun Posted April 24, 2013 Author Report Share Posted April 24, 2013 This is part of my log showing the problem:[20130324 17:03:26] File event /Volumes/Internal/radiot.bts/podcast.rss 67584[20130324 17:03:27] [OnNotifyFileRemove] /Volumes/Internal/radiot.bts/podcast.rss[20130324 17:03:38] SyncFilesController[file removed]: processing file removal /Volumes/Internal/radiot.bts/podcast.rss[20130324 17:03:40] Merge: will request file for /podcast.rss, ltime:1366841018 lhash:1F12910C3CA060F8C1ED1D1FF5D4B39C1261E4C3 rtime:1366847204 rhash:296249FF64DD38E444759F83DA79D2CBE3BB2501[20130324 17:03:40] SyncFilesController: Got file from remote (95.143.222.146:33333): podcast.rss state: 1 type: FT_FILE total:4 have:4 t:1366847204 mt:1366847198 4509A834A9353A3B79BE30B9F167AB0C19210DBC[20130324 17:03:41] Going to connect to peer 109.254.155.162:36778 for file podcast.rss[20130324 17:03:41] Torrent podcast.rss status: 137 error: <NULL> meta: 1[20130324 17:03:42] Torrent podcast.rss status: 137 error: <NULL> meta: 1[20130324 17:03:43] Torrent podcast.rss status: 137 error: <NULL> meta: 1[20130324 17:03:43] podcast.rss: Piece 0 complete[20130324 17:03:43] podcast.rss: Piece 1 complete[20130324 17:03:43] podcast.rss: Piece 3 complete[20130324 17:03:44] Torrent podcast.rss status: 137 error: <NULL> meta: 1[20130324 17:03:44] podcast.rss: Piece 2 complete[20130324 17:03:44] Finished downloading file podcast.rss, writing file attributes mt:1366847198[20130324 17:03:45] Torrent podcast.rss status: 137 error: <NULL> meta: 1[20130324 17:03:45] Force unloading torrent podcast.rss Quote Link to comment Share on other sites More sharing options...
umputun Posted April 24, 2013 Author Report Share Posted April 24, 2013 Another bit of info I just discovered - this file has wrong timestamp, something in the future. My local time is 5:51p now, but file shows 6:46p. Probably this is the reason why I can't delete/edit.I don't have any of my computers (admins) with wrong time or incorrect TZ, so not sure how this 6:46p was set. Obviously, I can't say the same about read-only remote clients. Quote Link to comment Share on other sites More sharing options...
GreatMarko Posted April 24, 2013 Report Share Posted April 24, 2013 Another bit of info I just discovered - this file has wrong timestamp, something in the future. My local time is 5:51p now, but file shows 6:46p. Probably this is the reason why I can't delete/edit.Yes, this will likely be the cause of your issue! - the timezone itself is less important (BitTorrent Sync converts all file timestamps to UTC for comparison anyway)I don't have any of my computers (admins) with wrong time or incorrect TZ, so not sure how this 6:46p was set. Obviously, I can't say the same about read-only remote clients.The issue likely won't be with the time on the remote "read-only" devices, as files/changes on "read-only" devices are essentially "ignored" - I'd double check the clock on all your full-access devices again! Quote Link to comment Share on other sites More sharing options...
umputun Posted April 24, 2013 Author Report Share Posted April 24, 2013 The issue likely won't be with the time on the remote "read-only" devices, as files/changes on "read-only" devices are essentially "ignored" - I'd double check the clock on all your full-access devices again!I double checked all my devices and clocks are fine.Anyway, if by some reason user get a file with such incorrect timestamp, that file becomes sort of "untouchable". Such behavior is very confusing and it is really unclear how to fix it on user's side. Maybe btsync shouldn't allow such timestamps in first place and as it see file in distant future (+5-10 minutes) it will reset modification time to the current local time? Quote Link to comment Share on other sites More sharing options...
GreatMarko Posted April 25, 2013 Report Share Posted April 25, 2013 Maybe btsync shouldn't allow such timestamps in first place and as it see file in distant future (+5-10 minutes) it will reset modification time to the current local time?BTSync needs a way to be able to compare files in order to determine which ones have changed/are more recent, etc. It does this taking into account the timestamps of the files. "Resetting" timestamps on files that differ by more than 5-10mins would cause all kinds of issues!That aside, your particular timestamp issue - if all your clocks are correct - is indeed a little strange!! Quote Link to comment Share on other sites More sharing options...
umputun Posted April 25, 2013 Author Report Share Posted April 25, 2013 Well, such time-only-based detection is extremely scary thing. It means if, by some reason, clocks on my computer not in full sync, or if I get from somewhere a file with incorrect timestamp it can cause all sort of interesting issues and lack of ability to delete won't be the worst thing. You can loose all updates of that file because btsync will overwrite it every time with "more recent" version.From another hand, if synced clock is so important for normal functioning of BTSync, it shouldn't even run in such situations but show some warning about time mismatch. As an example of such approach - as far as I remember some of AWS APIs are time-sensitive (for instance SQS) and won't accept request from devices with +/-300sec time offset.UPD: ok, this is not as bad as I thought. Looks like btsync normally ignores such file "from the future". Just tried to make a file with +5mins timestamp and btsync completely ignored this one and didn't sync till I put a valid timestamp. Quote Link to comment Share on other sites More sharing options...
GreatMarko Posted April 25, 2013 Report Share Posted April 25, 2013 Well, such time-only-based detection is extremely scary thing.Well, I'm basically going off the information the developers have previously provided in a previous thread, namely:"If file has the same name, Sync will detect the latest file, based on modification time, and distribute it across computers. Identical files will stay in folders and differences will be transferred between computers.""It converts time to GMT and then compare it. So it compares time in the same timezone." Quote Link to comment Share on other sites More sharing options...
AltJ Posted April 25, 2013 Report Share Posted April 25, 2013 I just ran into the same issue; due to a mis-configured client which had its time ahead a few hours. The bad thing is it won't allow you to even delete the file unless you manually delete it from all of the clients at the same time. I have verified this with Version 1.0.116 on both Linux & Mac. umputun, instead of making a file with a future timestamp, try setting your clock ahead, starting btsync then making the file. In my case the files were created with the clock an hour ahead while the client was disconnected (but btsync was running) Quote Link to comment Share on other sites More sharing options...
umputun Posted April 25, 2013 Author Report Share Posted April 25, 2013 The bad thing is it won't allow you to even delete the file unless you manually delete it from all of the clients at the same time.And if this file some sort of editable document/text - you will quietly loose all changes you made, which is in my opinion, the worst thing any sync system can do Probably those changes can be found in trash subdirectory, but still very bad behavior anyway. Quote Link to comment Share on other sites More sharing options...
graphicsmagick Posted April 26, 2013 Report Share Posted April 26, 2013 Seems to me that the BT Sync app desperately need a system clock verification mechanism built into it. At the very least, a big fat warning should be raised when the timestamps of a host doesn't match the others' (or the trackers') clocks.BTW, I want to know if BT Sync does checksum every file as part of the sync / transfer process? I see checksumming all chunks is one of the BitTorrent protocol's biggest strength. Quote Link to comment Share on other sites More sharing options...
umputun Posted April 29, 2013 Author Report Share Posted April 29, 2013 Just got another case. Somehow btsync thinks timestamp is 1408842928 (2014/08/23 14:22:44.000 GMT) and reverts back all my updates for these two file every time I try to touch it. All full-access boxes have correct clock. Quote Link to comment Share on other sites More sharing options...
Automatic Coding Posted April 30, 2013 Report Share Posted April 30, 2013 "It converts time to GMT and then compare it. So it compares time in the same timezone."I think he was more talking about if the base time was out, for instance if I have a computer that thinks today is 00:00:00 1st of January 1970 GMT, then, whatever timezone I turn it to (Hell, it can even add 12 hours to this and take 12 hours from the other timezone), the update will be overwritten by my other computer that thinks today is 1:21AM, 30th of April 2013 EST.Best bet would be to run an hourly/daily NTP polling system, although, I think most operation systems do this? I believe windows does it once a week (if NTP is set up), linux (or ubuntu, to be more exact. I've only really ever used ubuntu) has a program to update it (ntpupdate? ntppull? Something like that), which, I don't believe is automatically executed, routerOS has a NTP station that pulls minutely I think. Quote Link to comment Share on other sites More sharing options...
Automatic Coding Posted April 30, 2013 Report Share Posted April 30, 2013 Just got another case. Somehow btsync thinks timestamp is 1408842928 (2014/08/23 14:22:44.000 GMT) and reverts back all my updates for these two file every time I try to touch it. All full-access boxes have correct clock.Best bet would probably to copy the file out of the sync, then run "touch" on it, then copy it back in.But before you do that, can you run the following command on it?stat $filenameE.G.:-stat rssproxy/podcast.rssIt should give you something like this:-automatic@automatic-G74Sx:~/Sync$ stat winbox.exeFile: `winbox.exe'Size: 125440 Blocks: 248 IO Block: 4096 regular fileDevice: 801h/2049d Inode: 24642583 Links: 1Access: (0644/-rw-r--r--) Uid: ( 1000/automatic) Gid: ( 128/vboxusers)Access: 2013-04-28 23:30:23.661197107 -0400Modify: 2013-04-26 10:11:02.192668120 -0400Change: 2013-04-26 10:11:02.192668120 -0400Birth: -Can you give us what it says for said file? Quote Link to comment Share on other sites More sharing options...
umputun Posted April 30, 2013 Author Report Share Posted April 30, 2013 Can you give us what it says for said file?File in question is not podcast.rss but rssproxy.py and readme.$stat rssproxy.py16777223 9381841 -rwxr-xr-- 1 umputun staff 0 1149 "Apr 29 12:56:51 2013" "Aug 23 09:20:59 2014" "Apr 29 12:56:10 2013" "Apr 26 16:28:44 2013" 4096 8 0 rssproxy.pyBest bet would probably to copy the file out of the sync, then run "touch" on it, then copy it back in.Not sure I understand what you suggest. I have already tried to modify this file but it got reversed by btsync. Probably if I touch it with timestamp > 1408842928 I will able to update and then delete. Quote Link to comment Share on other sites More sharing options...
Automatic Coding Posted April 30, 2013 Report Share Posted April 30, 2013 File in question is not podcast.rss but rssproxy.py and readme.$stat rssproxy.py16777223 9381841 -rwxr-xr-- 1 umputun staff 0 1149 "Apr 29 12:56:51 2013" "Aug 23 09:20:59 2014" "Apr 29 12:56:10 2013" "Apr 26 16:28:44 2013" 4096 8 0 rssproxy.pyNot sure I understand what you suggest. I have already tried to modify this file but it got reversed by btsync. Probably if I touch it with timestamp > 1408842928 I will able to update and then delete.That's why I said copy it out of the btsync directory (so it's 'deleted'), touch it, then copy it back in.Anyway, it's not really BTSync's fault as the file technically was last modified at Aug 23, 2014. Quote Link to comment Share on other sites More sharing options...
umputun Posted April 30, 2013 Author Report Share Posted April 30, 2013 That's why I said copy it out of the btsync directory (so it's 'deleted'), touch it, then copy it back in.You can't delete such file without stopping sync on all clients. Anyway, it's not really BTSync's fault as the file technically was last modified at Aug 23, 2014.I think it is. The only program touched this file except my text editor was btsync. I think there is a bug in btsync causing incorrect timestamp to be assigned to some random files. Quote Link to comment Share on other sites More sharing options...
Automatic Coding Posted April 30, 2013 Report Share Posted April 30, 2013 You can't delete such file without stopping sync on all clients.I'm confused, yes, it'll stop syncing that file to all clients (& be deleted on those clients), but, it'll be temporary (For about 10 seconds while you touch it and copy it back in), unless I'm missing something? Quote Link to comment Share on other sites More sharing options...
umputun Posted April 30, 2013 Author Report Share Posted April 30, 2013 I'm confused, yes, it'll stop syncing that file to all clients (& be deleted on those clients), but, it'll be temporary (For about 10 seconds while you touch it and copy it back in), unless I'm missing something?Well, I also can wait till Aug 23, 2014 and problem will resolve itself But this is the second case I have been observing for last 3 days and I don't think the question "how to delete such file" is critical here. The real problem to be resolved by devs - how to prevent such behavior of btsync. Quote Link to comment Share on other sites More sharing options...
Automatic Coding Posted April 30, 2013 Report Share Posted April 30, 2013 Well, I also can wait till Aug 23, 2014 and problem will resolve itself But this is the second case I have been observing for last 3 days and I don't think the question "how to delete such file" is critical here. The real problem to be resolved by devs - how to prevent such behavior of btsync.I'm not saying that the resolution is to delete it (I'm not even saying that's a work around, I said move it out, update it, then move it back in), however, I am giving you a work-around.As for the issue itself, I can't comment, I've not had any issues like that. Quote Link to comment Share on other sites More sharing options...
AltJ Posted April 30, 2013 Report Share Posted April 30, 2013 I opened a bug with bittorrent and this issue has now been fixed! http://syncapp.bittorrent.com/1.0.125/ Quote Link to comment Share on other sites More sharing options...
umputun Posted April 30, 2013 Author Report Share Posted April 30, 2013 I opened a bug with bittorrent and this issue has now been fixed!http://syncapp.bittorrent.com/1.0.125/Which part of the problem fixed? As I see files from the future act exactly the same (untouchable). Is there a list of changes for 1.0.125 somewhere ? Quote Link to comment Share on other sites More sharing options...
Automatic Coding Posted April 30, 2013 Report Share Posted April 30, 2013 Which part of the problem fixed? As I see files from the future act exactly the same (untouchable). Is there a list of changes for 1.0.125 somewhere ?The issue with them being untouchable won't be the thing that's patched, considering that's how BTSync determines which item is going to replace what. If anything is fixed, it'll be that it'll stop touching random files years into the future, although, I've not received this so I can't really state that it is BTSync that's doing that, but, if you've had it then I can't really refute.As for the changelog, if you do see a changelog, mind linking me it? Quote Link to comment Share on other sites More sharing options...
AltJ Posted April 30, 2013 Report Share Posted April 30, 2013 For me, I was able to delete or edit a file that had a timestamp 2 hours into the future and have that change be synced around. I updated to 1.0.125 on all of my client machines.So, the files were no longer untouchable in my case. Quote Link to comment Share on other sites More sharing options...
umputun Posted April 30, 2013 Author Report Share Posted April 30, 2013 Good news. Probably in my case a lot of remote read-only clients on 1.0.116 won't allow to delete. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.