What limits speed?

[Rootyb]

So, in watching a few of my transfers, I notice that neither of my connections are getting anywhere close to maxing out either of the connections.

The upload connection is on a 500mbit connection. The download connection is "up to" 50mbit.

Pretty regularly, I can get up to 5-6MB/sec to/from these devices using LFTP's pget (I use -n 50, but it tends to max out the performance increase at around -n 35ish).

These are large (1GB+) files.

Is there anything I can do to to boost the speeds a bit?

Also, I notice that only one file transfers at a time (or at least, one file showing in the "Transfers" page). Is that normal? I've seen people mention multiple simultaneous downloads.

[GreatMarko]

Is there anything I can do to to boost the speeds a bit?

Also, I notice that only one file transfers at a time (or at least, one file showing in the "Transfers" page). Is that normal? I've seen people mention multiple simultaneous downloads.

Is BitTorrent Sync establishing direct, or relayed connections to your other devices? (direct connections would be faster!) - to determine this, if you look in the "Devices" tab, the icon adjacent to each device indicates how it's connected. A two-way arrow denotes a direct connection, a "cloud" icon indicates an indirect (relayed) connection

BitTorrent Sync is capable of transferring multiple files simultaneously, so it's a little strange that you're only seeing one at a time appear in the "Transfers" tab.

It may be a silly question, but you've not limited the download/upload rate (on the "Preferences" tab) have you? (a value of 0 = no limits)

[Rootyb]

Hmmm... Two arrows, so a direct connection.

Download/upload rate limit is set to 0 on both ends.

One change I made to settings was changing "disk_low_priority" to false on the download end, trying to improve the speed issues (no real change).

[kos13]

So, in watching a few of my transfers, I notice that neither of my connections are getting anywhere close to maxing out either of the connections.

I am not sure that I get all the numbers. So you trying to synchronize two machines. Your uplink is 500Mbs download link for second machine is 50Mbs. You are sending 1G file. What speed you see?

Yes, 1 connection per file is normal.

[GreatMarko]

One change I made to settings was changing "disk_low_priority" to false on the download end, trying to improve the speed issues (no real change).

If your devices are on the same LAN, you could also try setting "lan_encrypt_data" to "false" and "lan_use_tcp" to "true". This can potentially increase the speed of sync on low-end devices due to lower use of CPU

Also, check that "rate_limit_local_peers" is set to false

(You'll need to restart BitTorrent Sync for these advanced pref changes to take affect)

[Rootyb]

Not a local server.

I've got BT Sync (linux) installed on a server with a 500mbit connection. It has three folders shared.

I've got a machine (Windows 7) at home running BT Sync. It's the recipient (using the read-only secret of each). My home connection is 50mbit.

I'm getting speeds of about 2-3MB/sec. Completely not bad, but I'd love to squeeze out a couple more MB/sec if possible.

kos, I'm not getting one per file. I'm only getting one entry at a time in the transfer window, period. That is, even when I have multiple files, only one appears to be downloading at a time. One finishes, the next starts up.

[kos13]

We are working on optimizing protocol further, so please wait for next builds. We will be faster.

If the file is big, we transfer only one file in one connection. We are going to make in configurable. So transferring two files most likely will consume all bandwidth you have.

[Rootyb]

Awesome. Thanks for the update!

Great program, btw. As I mentioned in my other thread, I hadn't been able to find something easy, cross-platform, and that would copy files one-way, just once (so I could move the local copy without it constantly redownloading). I was literally learning python so I could build something myself. (I'm still learning, but just for fun, now).

Thanks!

[Automatic Coding]

I've also noticed that on my gigabit LAN network I only seem to transfer at ~ 20Mb/s (2.5MB/s).

Network testing I pull ~ 985Mb/s (123MB/s), the read speed of the submitter is ~ 80MB/s and the write speed of the receiver is ~ 60MB/s.

Too be honest, it's not that bad, as, I rarely ever update my files, and, if I do, oh well, it takes a few more minutes.

[kos13]

Sync uses adaptive uTP, so if there is any traffic it will reduce the speed of the transmission. We do work on improving the speed for 1:1 connection, stay tuned.

[yottabit]

The upload connection is on a 500mbit connection. The download connection is "up to" 50mbit.

[...]

Pretty regularly, I can get up to 5-6MB/sec to/from these devices [...]

6 MB/s = 6 * 8 = 48 Mbps ... so you're getting very close to your max download, which is imo remarkable given the encryption overhead and various other things that could be contending for time on your Internet connection.

Later in the thread you mentioned 2-3 MB/s, which is still not bad, depending what else you have going on.

I saw a ~huge~ increase in speed on a direct LAN transfer after disabling the encryption. I noticed it was bogging down on the virtualized Linux server since btsync appears to be single-threaded and was maxing out one core.

[kos13]

Encryption is very painful in terms of speed. Disabling it for LAN should increase the speed.

[Rootyb]

6 MB/s = 6 * 8 = 48 Mbps ... so you're getting very close to your max download, which is imo remarkable given the encryption overhead and various other things that could be contending for time on your Internet connection.

Later in the thread you mentioned 2-3 MB/s, which is still not bad, depending what else you have going on.

I saw a ~huge~ increase in speed on a direct LAN transfer after disabling the encryption. I noticed it was bogging down on the virtualized Linux server since btsync appears to be single-threaded and was maxing out one core.

Yeah, LFTP is kind of amazing. It saturates my connection (which, for what I'm doing, is exactly what I want it to do).

[yottabit]

FTP can be a fairly fast protocol for sure, especially when running parallel transfers. FTP's primary disadvantages include:

• An old-school, ill-conceived design separating the control channel and the data channel, which causes numerous NAT problems due to random port allocation for the (reverse-direction) of the data channel
  
• No built-in compression for compressible files
  
• Insecure (cleartext!) authentication by default, and no encryption for data channel
  
• Some hacked up FTP can use SSL/TLS for authentication, but it's a real pain to setup because so many servers/clients do it in so many ways
  

I really like that BTSync seems to "just work" in almost all cases, includes strong encryption by default, uses the venerable p2p protocol technology from BT, and has some options to disable encryption on the LAN when warranted.

And it's alpha! It will only get better from here!

[iamjohnny]

I'm running BTSync on a quite fast server too (about 500mbps but it's shared) and I usually max out my home connection (35mbps) with lftp and -n 10.

Now I get nice speeds most of the time with BTSync, ~4.4MB/s which is my absolute maximum. But the speed drops occasionally, down to 1MB/s, up to 2MB/s, down to 1MB/s and back to 4.4MB/s. I usually don't have that when using lftp.

[phant]

What can I say, I have an average speed of 60 kB/s... and my network is 100/100 (local server). And the connection is direct,,,

[kos13]

this is strange. Could you please follow these steps

[rdebath]

One thing I've noticed is that the use of multicast packets seems a little fragile.

I've had the Windows client stop sending or responding to multicast (no packets on port 3838), couldn't repeat it but that time the connections went via a remote relay. (Checking for local lan connections was turned on, and frobbed )

I have a Linux bridging setup where the iptables firewall on the bridge was changing the source address of the mutlicast packets to the firewall's internal address. This caused btsync to attempt to connect to the firewall not the across the bridge to the other client. Once multicast was excluded from the MASQ rules it worked fine.

You may have better luck using either (or both of) the limited broadcast or the various subnet broadcast addresses for the connected networks.

[tailor]

Does BTSync compress it's traffic, ie if I share a wav that is 100mb would it have to transfer the 100mb of data or does it use compression, like a webserver uses gzip I guess?

[rdebath]

BTSync does not use compression.

IMO, as it uses encryption it should have the ability to use compression (it stops any later compression happening) but it must be applied carefully as it's an expensive operation and likely to be unnecessary with the large files that BTSync has a preference for. The encryption can already cause a significant limit on the top speed without adding the cost of compressing too.

[gmongell]

What are the limitations on the encryption rate?  Is bit-torrent sync capable of using GPUs to increase the encryption/decryption rates?  Also, is bit-torrent sync available for non-GUI Linux environments?  There is not a good way to communicate between Linux/windows hosts on a continuous basis and bit-torrent sync might be able to master that niche.  Additionally, how does bit-torrent sync play with VPNs?

[scintilla13]

If your devices are on the same LAN, you could also try setting "lan_encrypt_data" to "false" and "lan_use_tcp" to "true". This can potentially increase the speed of sync on low-end devices due to lower use of CPU

Also, check that "rate_limit_local_peers" is set to false

(You'll need to restart BitTorrent Sync for these advanced pref changes to take affect)

 

This worked for me today without restarting BitTorrent. Now it can take advantage of 100Mbps LAN instead then 2 Mbps previous usage. Now I'm joking a little: to optimize bandwidth usage, next file processing and buffering should be prepared by a background thread ...

Fantastic Work .. in many applications dropbox has no more reasons to exists... 

[scintilla13]

Since "lan_use_tcp" option was deprecated, I was no more able to take advantage of LAN speed sync

[RomanZ]

@scintilla13

The option was deprecated as Sync now always prefers TCP if possible. So consider it to be turned on all the time. If you are experiencing some troubles with syncing speed over LAN - I suggest opening support ticket for us and including profiler information.