Two of my shares won't sync completely

[hawibtsync]

I saw similar messages here in this forum but I would like to help you to track that down. Why do several of my shares show differences? How to find those differences and how to get a better feeling that BTSync is working correct ;-)

 

 

General

=======

 

2x Windows 8.1, all on version 1.1.82

4x Android (3 different device types, 3 different versions), all on version 1.1.33

2x Linux i386, all on version 1.1.82

 

All these devices are working in my own LAN, no WAN connection.

 

I painfully avoid foreign characters in all my folders.

 

 

State

====

 

1x share shows 2.9 MB difference with an upload arrow (from Windows&Linux to Android)

1x share shows 8.0 MB difference with an upload arrow (from Windows&Linux to Android)

1x share shows 46.9 MB difference with an upload arrow (from Linux to Windows)

 

These differences didn't go away in a month now.

 

Time difference according to sync.log -8.

 

Debug is switched on (> 100MB). in this sync.log file I can't find any hints why there are differences.

 

I did compare all devices with TotalCommanders Sync-Directories-Feature. The result:

 

* The first two shares shown above (Windows&Linux to Android) are in sync (but don't show so).

* The third share shown above (Linux to Windows) is different (but not on 46.9 MB).

 

 

Help needed

==========

 

Currently I don't trust the BTSync because of the differences I see. So, once a week I use the TotalCommander sync-tool to compare all directories on all devices and there's always something I have to fix. Sometimes I need to remove a file on all devices and to re-copy it back, sometimes I simply copy a missing file, lots of times there are *!sync* files to remove, and and and ...

 

So what can I do to help you to fix these errors. A full blown DEBUG sync.log is sitting here. Any interest?

 

Thanks in advance.

 

[Osiris]

Go all open security-wise on file level & see what gives ...

Adapt accordingly & give full access to 'others' on files for both btsync program as synced folder (& all files & folders below).

chmod -R 777 /mnt/raid6/programs/btsync/chmod -R 777 /mnt/raid6/btsynced

Also, go all open in samba/cifs -> creation masks 0777 or 2777

 

If you see a sudden boost in sync speed, this will show I was right on a few previous accounts.

It's a rights problem.

 

Do me a favor. Post the owners of your synced files as well as your .SyncBlabla folders/files.

Is your btsync user 'btsync' the owner of the .Sync... folders/files ?

Perhaps adapt this as well with a chown btsync:nogroup (or the right group) on the .Sync folders/files.

 

Do this for linux & android ...

[hawibtsync]

Thanks for your answer.

 

99,99999% of all folder/file creation/update/delete is done on one of my Windows machines. This is replicated to the other devices (some have all shares, some only few). So I doubt that permissions might produce problems here.

 

However, I don't have root access to the four Android devices. All stock Android (Galaxy Nexus, etc.). I did install the APK and that's all. Configuration was done with the help of the BTSync settings. No manual interaction here.

 

The two Linux machines are the only ones where permissions might come into the game. But these are unRAID servers (that's a special Slackware distribution created by www.lime-technology.com). Again, i don't use command line here. btsync is a package for this distribution. I did install that package and configured it with the BTSync settings.

 

I checked some things on the Linux machines. User is "nobody:users". The btsync process is running under that same user:

nobody    2615     1  3 13:59 ?        00:04:28 /usr/local/btsync/btsync --config /mnt/cache/.btsync/btsync.conf

I do see some permission differences but in all cases nobody is the owner:

 

 

519156  909 -rw-r--r--  1 nobody users  927851 2005-11-27 10:12 PICT1003.JPG519141 1161 -rwxr-xr-x  1 nobody users 1186485 2005-11-27 10:13 PICT1004.JPG

I don't understand it...

[hawibtsync]

Ok, I found it. I had to compare every device combination individually with my "TotalCommander Sync Directory feature" to find the differences. In 100% it was a case mismatch. I found something like that on one of the Linux i386 boxes:

A document.pdfA Document.pdf

I could reproduce it as well. You need a mix of different OS platforms to catch that problem:

 

* On a Windows box create a file like the first one above "A document.pdf"

 

Now the other devices start to sync this file. Rename that file to e.g. "A Document.pdf". It is very important to do this while some devices did sync already but a slower device is in the middle of that operation.

 

 

There are two possible results:

 

* On one of the devices you see a *!sync* file. Means something has been broken. The status within the BTSync GUI will show a device with pending transfer volumes (it will never complete).

 

* You see both files ("A document.pdf" and "A Document.pdf") on one of the devices that has a case sensitive filesystem.

 

You won't see this problem if you wait for all devices to complete the first sync and rename later.

[Osiris]

You cannot begin to imagine how much this has helped me.

I ran this on both machines

find /mnt/raid6/video/ -name "*.\!sync" -exec rm {} \;

and restarted the btsync processes.

Syncing resumed.

 

a massive +1, if u ask me.

 

I'm talking about a sync between a freebsd (Freenas 9) and my qnap.

Had to run the line on a mapped drive from my ubuntu server to the qnap nas, since their find command doesn't allow the -exec.

So In my opinion, case sensitivity has nothing to do with it.

 

It also means that everything I said before was wrong or insignificant. Hurrah !

[Osiris]

Update. Still syncing. Less than a terrabyte to go.

 

Where can I find info on the whole syncing process ? Can't seem to find it in the docs.

 

It seems some .!sync files are still being created on my read-only destinations.

I assume these files are to be considered in progress.

 

Yesterday, there were different ones on my destination but also 1 on my source.

This is weird since there's no two way syncing being done.

I'm assuming that syncing of the (large) file started while the file was being copied to the system and the .!sync file was then created somehow.

I'm going to prudently conclude that this one file was the only problem.