Mixed sync, delete file issue

[umputun]

Hi,

I have some strange behavior and to me it looks like a bug. I have a folder shared with read-only secret across ~100 users. At the same time it shared with full-access secret for some limited number of "admins". Each admin can add new file - this part works just fine. However it seems to be impossible to remove a file - even for admins it acts like "read-only", i.e. removed file get restored automatically.

Is this expected behavior or unexpected bug? Any idea how file can be deleted, maybe some workaround?

[umputun]

This is part of my log showing the problem:

[20130324 17:03:26] File event /Volumes/Internal/radiot.bts/podcast.rss 67584
[20130324 17:03:27] [OnNotifyFileRemove] /Volumes/Internal/radiot.bts/podcast.rss
[20130324 17:03:38] SyncFilesController[file removed]: processing file removal /Volumes/Internal/radiot.bts/podcast.rss
[20130324 17:03:40] Merge: will request file for /podcast.rss, ltime:1366841018 lhash:1F12910C3CA060F8C1ED1D1FF5D4B39C1261E4C3 rtime:1366847204 rhash:296249FF64DD38E444759F83DA79D2CBE3BB2501
[20130324 17:03:40] SyncFilesController: Got file from remote (95.143.222.146:33333): podcast.rss state: 1 type: FT_FILE total:4 have:4 t:1366847204 mt:1366847198 4509A834A9353A3B79BE30B9F167AB0C19210DBC
[20130324 17:03:41] Going to connect to peer 109.254.155.162:36778 for file podcast.rss
[20130324 17:03:41] Torrent podcast.rss status: 137 error: <NULL> meta: 1
[20130324 17:03:42] Torrent podcast.rss status: 137 error: <NULL> meta: 1
[20130324 17:03:43] Torrent podcast.rss status: 137 error: <NULL> meta: 1
[20130324 17:03:43] podcast.rss: Piece 0 complete
[20130324 17:03:43] podcast.rss: Piece 1 complete
[20130324 17:03:43] podcast.rss: Piece 3 complete
[20130324 17:03:44] Torrent podcast.rss status: 137 error: <NULL> meta: 1
[20130324 17:03:44] podcast.rss: Piece 2 complete
[20130324 17:03:44] Finished downloading file podcast.rss, writing file attributes mt:1366847198
[20130324 17:03:45] Torrent podcast.rss status: 137 error: <NULL> meta: 1
[20130324 17:03:45] Force unloading torrent podcast.rss

[umputun]

Another bit of info I just discovered - this file has wrong timestamp, something in the future. My local time is 5:51p now, but file shows 6:46p. Probably this is the reason why I can't delete/edit.

I don't have any of my computers (admins) with wrong time or incorrect TZ, so not sure how this 6:46p was set. Obviously, I can't say the same about read-only remote clients.

[GreatMarko]

Another bit of info I just discovered - this file has wrong timestamp, something in the future. My local time is 5:51p now, but file shows 6:46p. Probably this is the reason why I can't delete/edit.

Yes, this will likely be the cause of your issue! - the timezone itself is less important (BitTorrent Sync converts all file timestamps to UTC for comparison anyway)

I don't have any of my computers (admins) with wrong time or incorrect TZ, so not sure how this 6:46p was set. Obviously, I can't say the same about read-only remote clients.

The issue likely won't be with the time on the remote "read-only" devices, as files/changes on "read-only" devices are essentially "ignored" - I'd double check the clock on all your full-access devices again!

[umputun]

The issue likely won't be with the time on the remote "read-only" devices, as files/changes on "read-only" devices are essentially "ignored" - I'd double check the clock on all your full-access devices again!

I double checked all my devices and clocks are fine.

Anyway, if by some reason user get a file with such incorrect timestamp, that file becomes sort of "untouchable". Such behavior is very confusing and it is really unclear how to fix it on user's side. Maybe btsync shouldn't allow such timestamps in first place and as it see file in distant future (+5-10 minutes) it will reset modification time to the current local time?

[GreatMarko]

Maybe btsync shouldn't allow such timestamps in first place and as it see file in distant future (+5-10 minutes) it will reset modification time to the current local time?

BTSync needs a way to be able to compare files in order to determine which ones have changed/are more recent, etc. It does this taking into account the timestamps of the files. "Resetting" timestamps on files that differ by more than 5-10mins would cause all kinds of issues!

That aside, your particular timestamp issue - if all your clocks are correct - is indeed a little strange!!

[umputun]

Well, such time-only-based detection is extremely scary thing. It means if, by some reason, clocks on my computer not in full sync, or if I get from somewhere a file with incorrect timestamp it can cause all sort of interesting issues and lack of ability to delete won't be the worst thing. You can loose all updates of that file because btsync will overwrite it every time with "more recent" version.

From another hand, if synced clock is so important for normal functioning of BTSync, it shouldn't even run in such situations but show some warning about time mismatch. As an example of such approach - as far as I remember some of AWS APIs are time-sensitive (for instance SQS) and won't accept request from devices with +/-300sec time offset.

UPD: ok, this is not as bad as I thought. Looks like btsync normally ignores such file "from the future". Just tried to make a file with +5mins timestamp and btsync completely ignored this one and didn't sync till I put a valid timestamp.

[GreatMarko]

Well, such time-only-based detection is extremely scary thing.

Well, I'm basically going off the information the developers have previously provided in a previous thread, namely:

"If file has the same name, Sync will detect the latest file, based on modification time, and distribute it across computers. Identical files will stay in folders and differences will be transferred between computers."

"It converts time to GMT and then compare it. So it compares time in the same timezone."

[AltJ]

I just ran into the same issue; due to a mis-configured client which had its time ahead a few hours.

The bad thing is it won't allow you to even delete the file unless you manually delete it from all of the clients at the same time.

I have verified this with Version 1.0.116 on both Linux & Mac.

umputun, instead of making a file with a future timestamp, try setting your clock ahead, starting btsync then making the file. In my case the files were created with the clock an hour ahead while the client was disconnected (but btsync was running)

[umputun]

The bad thing is it won't allow you to even delete the file unless you manually delete it from all of the clients at the same time.

And if this file some sort of editable document/text - you will quietly loose all changes you made, which is in my opinion, the worst thing any sync system can do Probably those changes can be found in trash subdirectory, but still very bad behavior anyway.

[graphicsmagick]

Seems to me that the BT Sync app desperately need a system clock verification mechanism built into it. At the very least, a big fat warning should be raised when the timestamps of a host doesn't match the others' (or the trackers') clocks.

BTW, I want to know if BT Sync does checksum every file as part of the sync / transfer process? I see checksumming all chunks is one of the BitTorrent protocol's biggest strength.

[umputun]

Just got another case. Somehow btsync thinks timestamp is 1408842928 (2014/08/23 14:22:44.000 GMT) and reverts back all my updates for these two file every time I try to touch it. All full-access boxes have correct clock.

[Automatic Coding]

"It converts time to GMT and then compare it. So it compares time in the same timezone."

I think he was more talking about if the base time was out, for instance if I have a computer that thinks today is 00:00:00 1st of January 1970 GMT, then, whatever timezone I turn it to (Hell, it can even add 12 hours to this and take 12 hours from the other timezone), the update will be overwritten by my other computer that thinks today is 1:21AM, 30th of April 2013 EST.

Best bet would be to run an hourly/daily NTP polling system, although, I think most operation systems do this? I believe windows does it once a week (if NTP is set up), linux (or ubuntu, to be more exact. I've only really ever used ubuntu) has a program to update it (ntpupdate? ntppull? Something like that), which, I don't believe is automatically executed, routerOS has a NTP station that pulls minutely I think.

[Automatic Coding]

Just got another case. Somehow btsync thinks timestamp is 1408842928 (2014/08/23 14:22:44.000 GMT) and reverts back all my updates for these two file every time I try to touch it. All full-access boxes have correct clock.

Best bet would probably to copy the file out of the sync, then run "touch" on it, then copy it back in.

But before you do that, can you run the following command on it?

stat $filename

E.G.:-

stat rssproxy/podcast.rss

It should give you something like this:-

automatic@automatic-G74Sx:~/Sync$ stat winbox.exe
File: `winbox.exe'
Size: 125440 Blocks: 248 IO Block: 4096 regular file
Device: 801h/2049d Inode: 24642583 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 1000/automatic) Gid: ( 128/vboxusers)
Access: 2013-04-28 23:30:23.661197107 -0400
Modify: 2013-04-26 10:11:02.192668120 -0400
Change: 2013-04-26 10:11:02.192668120 -0400
Birth: -

Can you give us what it says for said file?

[umputun]

Can you give us what it says for said file?

File in question is not podcast.rss but rssproxy.py and readme.

$stat rssproxy.py
16777223 9381841 -rwxr-xr-- 1 umputun staff 0 1149 "Apr 29 12:56:51 2013" "Aug 23 09:20:59 2014" "Apr 29 12:56:10 2013" "Apr 26 16:28:44 2013" 4096 8 0 rssproxy.py

Best bet would probably to copy the file out of the sync, then run "touch" on it, then copy it back in.

Not sure I understand what you suggest. I have already tried to modify this file but it got reversed by btsync. Probably if I touch it with timestamp > 1408842928 I will able to update and then delete.

[Automatic Coding]

File in question is not podcast.rss but rssproxy.py and readme.

$stat rssproxy.py
16777223 9381841 -rwxr-xr-- 1 umputun staff 0 1149 "Apr 29 12:56:51 2013" "Aug 23 09:20:59 2014" "Apr 29 12:56:10 2013" "Apr 26 16:28:44 2013" 4096 8 0 rssproxy.py

Not sure I understand what you suggest. I have already tried to modify this file but it got reversed by btsync. Probably if I touch it with timestamp > 1408842928 I will able to update and then delete.

That's why I said copy it out of the btsync directory (so it's 'deleted'), touch it, then copy it back in.

Anyway, it's not really BTSync's fault as the file technically was last modified at Aug 23, 2014.

[umputun]

That's why I said copy it out of the btsync directory (so it's 'deleted'), touch it, then copy it back in.

You can't delete such file without stopping sync on all clients.

Anyway, it's not really BTSync's fault as the file technically was last modified at Aug 23, 2014.

I think it is. The only program touched this file except my text editor was btsync. I think there is a bug in btsync causing incorrect timestamp to be assigned to some random files.

[Automatic Coding]

You can't delete such file without stopping sync on all clients.

I'm confused, yes, it'll stop syncing that file to all clients (& be deleted on those clients), but, it'll be temporary (For about 10 seconds while you touch it and copy it back in), unless I'm missing something?

[umputun]

I'm confused, yes, it'll stop syncing that file to all clients (& be deleted on those clients), but, it'll be temporary (For about 10 seconds while you touch it and copy it back in), unless I'm missing something?

Well, I also can wait till Aug 23, 2014 and problem will resolve itself But this is the second case I have been observing for last 3 days and I don't think the question "how to delete such file" is critical here. The real problem to be resolved by devs - how to prevent such behavior of btsync.

[Automatic Coding]

Well, I also can wait till Aug 23, 2014 and problem will resolve itself But this is the second case I have been observing for last 3 days and I don't think the question "how to delete such file" is critical here. The real problem to be resolved by devs - how to prevent such behavior of btsync.

I'm not saying that the resolution is to delete it (I'm not even saying that's a work around, I said move it out, update it, then move it back in), however, I am giving you a work-around.

As for the issue itself, I can't comment, I've not had any issues like that.

[AltJ]

I opened a bug with bittorrent and this issue has now been fixed!

http://syncapp.bittorrent.com/1.0.125/

[umputun]

I opened a bug with bittorrent and this issue has now been fixed!

http://syncapp.bittorrent.com/1.0.125/

Which part of the problem fixed? As I see files from the future act exactly the same (untouchable). Is there a list of changes for 1.0.125 somewhere ?

[Automatic Coding]

Which part of the problem fixed? As I see files from the future act exactly the same (untouchable). Is there a list of changes for 1.0.125 somewhere ?

The issue with them being untouchable won't be the thing that's patched, considering that's how BTSync determines which item is going to replace what. If anything is fixed, it'll be that it'll stop touching random files years into the future, although, I've not received this so I can't really state that it is BTSync that's doing that, but, if you've had it then I can't really refute.

As for the changelog, if you do see a changelog, mind linking me it?

[AltJ]

For me, I was able to delete or edit a file that had a timestamp 2 hours into the future and have that change be synced around. I updated to 1.0.125 on all of my client machines.

So, the files were no longer untouchable in my case.

[umputun]

Good news. Probably in my case a lot of remote read-only clients on 1.0.116 won't allow to delete.