What Speed Should I Expect? Folder With 30 Files @ 500Mib ... 10 Minutes...

[soon_dk]

I'm trying BTsync out on a local 100-1000 network for starters ... 1 desktop PC, one laptop and one Raspberry Pi. The setup works (on my LAN at least) ...but speed is disappointing...

 

If I add a folder to the sync folder on the PC with 30 files at a total of 500Mib it takes just over 10 minutes to sync to the laptop (both are fast machines) ... seems very slow to me ... and I would have thought, that the distributed nature of torrent syncing would mean that the slow element in my network (the Raspberry Pi) would not slow down syncing between the other two ...

 

Does this speed sound reasonable to you?

 

 

Linux on PC: i7 @ 3.3Ghz with 16 gb RAM wired network, Laptop i5 @ 2.5 Ghz with 8gb Ram wireless, Raspberry Pi (v2) wired network, Router Asus RT-AC68U

Edited May 14, 2014 by soon_dk

[RomanZ]

@soon_dk

 

No, the speed should be higher. Could you please try to disconnect RPI (or kill btsync running on RPI), restart other peers and see if they will saturate your bandwidth?

[soon_dk]

Can do ... I just need to re-install linux mint on my laptop ... :-)

[soon_dk]

I pulled the network from the RPI,

copied a 964 MiB iso image to the sync folder on my laptop and

deleted an empty folder in my desktop sync folder

 

Using KDE system monitor:

     on the laptop : hardly any network activity at all

     on the desktop: regular periodic peaks

...but nothing above 4 kb/s on either

...after 5 minutes - nothing has changed in the sync folders yet ...

[soon_dk]

My mistake -- btsync wasn't runing on the desktop :-)

 

Network trafic is now about 6-8 MiB/s on both

I then plug the RPI on the network again ... move another large file into the laptop sync folder ... and ... nothing ... weird ..

[RomanZ]

@soon_dk

 

Do I understand correctly that after connecting back RPI device it does not get the file? What about desktop - does it download file from laptop?

[soon_dk]

It's a little odd

 

The following is a little long, I know but I went about it systematically from a new beginning...

 

To re-iterate the setup: a wifi connected laptop, a wired desktop and a wired RPI on a LAN (spec's are listed in the first post above)

 

I have installed the linux_arm BTsync v. 1.3.94 on the RPI on a fresh installation of Raspbian

I have installed the linux_x64(glibc 2.3) v. 1.3.94 on both the laptop and desktop on Linux Mint 16 (KDE)

Web-interface is running fine on all three and each lists the other two as connected devices. The Connected devices and Status does not seems to correctly reflect what is going on ... the last 'Synced at'-time (after a page refresh) could be 20 minutes ago, when in fact a change was synced 5 minutes ago..

 

BTSync settings (for all three devices):

+ Use UPnP port mapping

Download limit 0

Upload limit 0

 

Folder preferences (for all three devices):

+ Use relay server when required

+ Use tracker server

+ Search LAN

- Search DHT network

+ Store deleted files in SyncArchive

- Use predefined hosts

 

 

Test 1:

Desktop, laptop and RPI all on-line

Working on the RPI (by ssh from desktop):

a ) Creating an empty folder : synced to both desktop+laptop in 13 seconds

b ) Removing the empty folder again: synced to both desktop+laptop in 16 seconds

c ) Downloading a 106MiB file from the internet to the RPI (download complete after 31 seconds): sync to both desktop+laptop started at 51 sec's and completed in 1:45 (file.zip.!sync was evident in that period of time) -- a screendump of the desktop KDE system monitor is here: www.oneill.dk/test_1.png

d ) Deleting the file again from the RPI was synced to both desktop+laptop in 5 seconds

 

Test 2:

Desktop, laptop and RPI all on-line

Working on the laptop (wifi):

a ) Creating an empty folder: synced to both RPI+desktop in 9:40 minutes

b ) Removing the empty folder again: synced to both RPI+desktop in 10 minutes

c ) Downloading a 106MiB file from the internet to the laptop (download complete in 19 sec's): sync to both RPI+desktop ... no apparent activity at all for 8+ minutes and then it seemed to complete sync in no time at all (I did not see a file.zip.!sync, but just file.zip ... I did blink during the 8+ minutes, however :-)

d ) Deleting the file again from the laptop was synced to both RPI+desktop in 8:10 minutes

 

Test 3:

Desktop and laptop on-line, the RPI off-line (btsync was killed and as expected the RPI disappeared from the list of connected deviced on both desktop and laptop BTsync web interface)

Working on the laptop (wifi):

a ) Creating an empty folder: synced to desktop in 1:37 minutes

b ) Deleting the empty folder again: synced to the desktop in 9:45 minutes

c ) Downloading a 106MiB file from the internet to the laptop (download complete in 16 sec's): sync to desktop started at 8:04 and completed at 8:30

d ) Deleting the file again from the laptop was synced to desktop 18 minutes later

 

Test 4

Desktop and laptop on-line, the RPI off-line

Working on the desktop

a ) Creating a new empty folder on the desktop was synced to the laptop in 7:31

b ) Deleting the empty folder again from the desktop was synced to the laptop in 9:50

... I did not bother trying the large file sync again ...

 

Test 5 -- repeat of Test 1

Brought the RPI on-line again -- the RPI showing up as expected as a connected device on laptop and desktop

I brought the RPI back online before the folder deletion (point b in test 4 above) had synced from desktop-to-laptop : the RPI showed up in connected devices within 2 seconds, but this did not trigger a sync of the folder-deletion.

a ) Created an empty folder on the RPI ... this was not synced by 4 minutes

b ) Downloaded the 106MiB file from the internet to the RPI (download completed in 25 secs), the sync to laptop/desktop had not started by 4 minutes

c ) Created a new empty folder (with unique names) on each of the three devices simultanuously:

  - New_folder_desktop was synced to laptop in 1 minute

  - New_folder_laptop was syned to desktop in 2:20 minutes

  - At 4 minutes, New_folder_RTI had not synced to desktop, nor laptop and nothing had synced to the RPI either

 

...so in summary:

i) all changes are synced correctly (given enough time)

ii) actual file transfer seems quick enough -- irrespective of the devices involved, but

iii) local changes made to the RPI was initially synced to desktop/laptop quite quickly (Test 1)

iv) local changes made to the laptop or desktop takes a very long time to sync to others (irrespective of whether the RPI is online or not)

v) the apparent quick syns with local changes to the RPI seen in Test 1 was not evident at the Test 5

 

This makes me think: for some reason, a sync is not triggered by local changes on the laptop or desktop, but happens only at regular intervals several minutes apart (every 10 minutes?). By contrast, changes made to the RPIs sync folder did seem to trigger a sync, but only in the first test (Test 1), not after having been off/on-line

 

I don't know how BTsync actually works, but I assume there is some underlying mechanism which monitors changes in the designated sync-folder and not just a regular periodic check for changes, every 10 minutes...

 

The designated sync folders are all on ext4 partitions. On the RPI it is a separate partition (USB drive) mounted inside the users home folder (/home/user/sync/). On the desktop+laptop, the home folders are separate partitions. I doubt this has any influence on the situation.

 

I have also looked at my router, nothing in particular jumps at me -- seems reasonable enough and in any case, it's all on my LAN so far..

[RomanZ]

@soon_dk,

 

Thanks a lot for the detailed description. Briefly: glibc 2.3 is unable to receive file/folder change notifications, so only full folder rescan "notices" your sync folder changes - and it occurs once per 10 minutes.

 

In details: your Desktop and Laptop are using "Glibc 2.3" version of Sync. One of the major differences between glibc 2.3 and 2.4 was the ability of 2.4 to set file system notifications, so the very OS is going to notify subscribers that something has changed in the specified folders. Sync uses 2 mechanisms to spot changes: file system notifications (if available) and full folder rescan (occurring every 10 minutes by default). In your case only 2nd mechanism is working.

 

I'd advise to upgrade your glibc to 2.4 or newer - and get the version of Sync working with glibc 2.4.

[RomanZ]

@soon_dk

 

What about actual transfer speed when transfer starts - does it saturate the whole channel or some of bandwidth left unused?

[soon_dk]

That fits in very well with the observations.

So, i should chance to the 2.4 version instead, i guess...

I'm away from my PC and laptop for a little while ... I'll reporter om the actual download speed in about a week and try the 2.4 version then, as well.