[Solved] Sync Stopped After 118 Gb

[tomzooi]

Hello All,

I found btsync to be the app I am looking for to sync my desktop data with my laptop, I am running into a problem however. I run two systems, one is arch 64 on a large asus laptop which I use as my desktop. The other system is ubuntu x64 14.04. Both are newest versions. The installation guides I followed, configs and logs will be posted below, I will first define the problem.

After installing btsync everything worked quit well, so I left the two devices on overnight to be able to sync all the 222Gbs of data. Waking up in the morning I found that both machines were still on, the webinterface of btsync was responding but the syncing had stopped at 118.92 GBs, allthough there should be 222GBs in there. I restarted everything, but no luck. Took a look at the logs and come upon repeating error messages on both machines.

On arch I see messages similar like these:
 
 

[20140821 15:48:01.084] Incoming connection from 130.89.170.174:7331[20140821 15:52:03.523] NAT-PMP: Unable to map port with NAT-PMP.[20140821 15:52:20.516] UPnP: Device error "http://130.89.161.149:32469/DeviceDescription.xml": (110) offline (timed out)[20140821 15:52:25.768] UPnP: Device error "http://130.89.166.244:42725": (-2)

 


130.89.170.174 is my laptop device in this case, I set it to port 7331

On Ubuntu I find them similar like these:
 
 

[20140821 16:12:03.274] TorrentFile: Failed to create empty suffix for file "/storage/data/Tom/Pictures/Foto's/2012-7-4 CatchUp Clubvolley/Selection/jpeg/2012-07-04_18:55:32.jpg" - 22[20140821 16:12:03.274] SyncFilesController: failed to load torrent for file "/storage/data/Tom/Pictures/Foto's/2012-7-4 CatchUp Clubvolley/Selection/jpeg/2012-07-04_18:55:32.jpg"

 


config files and logs are found on
http://dump.tomzooi.com/sync/

I followed this ubuntu guide: https://www.digitalocean.com/community/tutorials/how-to-use-bittorrent-sync-to-synchronize-directories-in-ubuntu-12-04
And this guide on ARCH: https://wiki.archlinux.org/index.php/BitTorrent_Sync
to install btsync.

Hope anybody can help .

[GreatMarko]

It may seem an obvious question, but is there adequate free space on your devices?

[tomzooi]

Yes there is plenty .

[tomzooi]

Ok, Today I had to reinstall ubuntu due to a stupid mistake. I deleted the folder I wanted to sync and installed btsync again. Basically everything was the same, it stopped at the same amount of files it did last time, and the log file on my ubuntu machine has the same kind of error messages (could not create empty suffix and failed to load torrent file).

 

I checked the .syncignore which was empty for just a couple of files like thumbs.db and other OS stuff.

 

I am now wondering if somehow either my creation dates are wrong or I have ownership issues of some kind. I also added a file on both systems with different names to see if after this sync run they will still sync, I'll update you on that. I still appreciate any suggestion of (stupid) things I might have overlooked or done wrong .

 

//Edit: the new file I added on the ubuntu machine was synced to the arch machine, I changed it's name which got synced back to my ubuntu machine, howerver I haven't seen the new file I created on the arch machine yet...

 

//edit two: still no other file on my machine, I did a stat of two folders, of which the first did not sync and the second did:

[tom@tomtop Tom]$ stat Pictures/Foto\'s/2014-5-18\ TC\ SG\ Motor\ experimenten/  File: ‘Pictures/Foto's/2014-5-18 TC SG Motor experimenten/’  Size: 4096          Blocks: 8          IO Block: 4096   directoryDevice: 805h/2053d    Inode: 754824      Links: 1Access: (0777/drwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)Access: 2014-05-20 20:45:04.308616700 +0200Modify: 2014-05-20 20:43:28.660261000 +0200Change: 2014-05-20 20:43:28.660261000 +0200 Birth: -[tom@tomtop Tom]$ stat Pictures/Foto\'s/2012-7-1\ Hybrid\ Worlds\ Installation/  File: ‘Pictures/Foto's/2012-7-1 Hybrid Worlds Installation/’  Size: 8192          Blocks: 16         IO Block: 4096   directoryDevice: 805h/2053d    Inode: 118059      Links: 1Access: (0777/drwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)Access: 2012-07-03 17:20:26.769215000 +0200Modify: 2012-07-03 17:20:24.509345000 +0200Change: 2012-07-03 17:20:24.509345000 +0200 Birth: -

It looks like the entire directory structure synced, but not all the files inside. If I do the same stats for the files inside these directories, they look as identical as the stats of the directories themself, so far I have not managed to find any differences...

Edited August 22, 2014 by tomzooi

[tomzooi]

Ok, i've got some fresh intelligence:

I copied some dropbox folders into my storage as a backup, and I noticed they got synced right away. It seems that "older" files are synced, however newer ones are not (you can see the difference in time above). I figure this might be a permission problem, but I cannot see any permission differences. The timestamps seem valid so far, and I do not see the files in .SyncIgnore or in the log files, I still only see

[20140821 16:12:03.274] TorrentFile: Failed to create empty suffix for file "/storage/data/Tom/Pictures/Foto's/2012-7-4 CatchUp Clubvolley/Selection/jpeg/2012-07-04_18:55:32.jpg" - 22[20140821 16:12:03.274] SyncFilesController: failed to load torrent for file "/storage/data/Tom/Pictures/Foto's/2012-7-4 CatchUp Clubvolley/Selection/jpeg/2012-07-04_18:55:32.jpg"

like error messages on my ubuntu system. I even changed the configuration so to run the daemon as root:root as suggested here but that did not help.

 

I must say, the two directories are mounted from a ntfs disk, on the arch machine as follows:

/dev/sda5 /storage/2 ntfs-3g defaults 0 1

and on the ubuntu machine as follow:

UUID=1CBE01BE15476FCE  /storage/data  ntfs-3g  defaults,windows_names,locale=en_US.utf8  0 0

I am not sure if it has anything to do with it but who knows. I would like to check if all the permissions on the arch machine for the folder I am trying to sync are similar, as well as the timestamps are correct, but so far I can only do stats on singe files, if somebody knows a way to do this it would be nice to let me know .

My arch logs show mostly this btw:

[20140823 11:52:17.237] Incoming connection from 130.89.169.206:50567[20140823 11:52:45.173] Incoming connection from 130.89.169.206:40344[20140823 12:07:13.523] Incoming connection from 130.89.169.206:50567[20140823 12:07:13.580] Incoming connection from 130.89.169.206:50567[20140823 12:07:14.325] Incoming connection from 130.89.169.206:50567[20140823 12:07:14.326] Incoming connection from 130.89.169.206:50567[20140823 12:07:14.427] Extension: ipv4:[130.89.169.206] for '/storage/2/Tom/Documents/TextDocument.odt'[20140823 12:07:24.370] Incoming connection from 130.89.169.206:50567

The TextDocument.odt is a file I created on the ubuntu machine which I then changed in arch. This changed got succesfully synced back to the ubuntu machine. However, I still cannot make new files on the arch machine that are synced to the ubuntu machine.

 

[tomzooi]

hmm, apparantly I have two different issues here

one: when I create an empty file on the arch machine, the stat looks like this:

[tom@tomtop Tom]$ stat /storage/2/Tom/Documents/archnew.txt  File: ‘/storage/2/Tom/Documents/archnew.txt’  Size: 2             Blocks: 1          IO Block: 4096   regular fileDevice: 805h/2053d    Inode: 26674       Links: 1Access: (0777/-rwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)Access: 2014-08-23 15:57:02.885295000 +0200Modify: 2014-08-13 07:58:14.000000000 +0200Change: 2014-08-23 15:57:02.886273100 +0200 Birth: -

noticing that the modify date is far before the access/change date which is strange and I think that is why btsync ignores it somehow after doing a touch it appears to be good again.

 

second appears to be filenames. as soon as I change, copy or move a file named like this:

/storage/data/Tom/Pictures/Foto's/2012-7-4 CatchUp Clubvolley/Selection/jpeg/2012-07-04_19:28:34.jpg

to something like u.jpg, simply remove the :'s (2012-07-04_192834.jpg) in the same folder, it will sync without issues.

 

Are these known bugs of bittorrent sync? I think I will need to touch and rename I my files don't I?

 

Tom

[tomzooi]

Yay, I think I fixed it, I changed the filenames to legal windows names and removed windows_names from /etc/fstab under ubuntu, just so they would be the same., so

 

Basic solution: if you see the error messages from the first post on the "client" side, check your filenames and your /etc/fstab to see if the filenames are allowed, if not: fix (easiest way is to remove windows_names from /etc/fstab, remount/restart/reboot everything)

 

Another of my own problems fixed by yours truly, making a habit out of it lately...

 

Interesting error btw from the forum:

Warning: Illegal string offset 'edit_topic' in /home/bittorre/public_html/btadmn/applications/forums/sources/classes/post/classPost.php on line 2239

 

Might want to check on that