crazyk4952

Inconsistent Speed

Recommended Posts

@adinunzio10

Thanks for your feedback. We'll address the issue when we collect enough info. If we need more information - i'll drop requests to this topic or directly to you.

 

Sounds good! Thanks for the help, and I am always willing to assist / provide feedback. Drop a PM though as now that my issue is resolved, I probably wont be returning to this thread often. PM will go directly to my email. 

Share this post


Link to post
Share on other sites

I just setup Sync 2.0.82 on two machines on a LAN (one Linux, one Windows) and have been getting abysmal speeds, of between 3-4MB/sec on a Gigabit LAN.  Every now and then it will spike up to ~10-12MB/sec for maybe a second but otherwise maintains a slow average. These are large ~400MB files but would normally be several gigabytes in size (large uncompressed video files.)

 

???

Share this post


Link to post
Share on other sites

@ManChicken

Let's try to do the following:

1. Shut down Sync

2. Open your storage folder (%appdata%\BitTorrent Sync for Windows, .sync subfolder for Linux), thre will be a bunch of Sync files (DBs, configs, etc).

3. Create debug.txt file

4. Put next data to it:

00000000<return>

0000000A

5. Start Sync

6. Do the same on second peer.

 

This will force Sync to use TCP connection instead of UDP. Let me know if it helps.

Share this post


Link to post
Share on other sites

Hi, 

just read the thread. I have also been having serious speed issues syncing to a host across the internet. I tried everything and cursed BTsync to the moon and back.  I think I just confirmed in my case it was not BTsync's fault.It was really is a problem with UDP traffic being throttled (heavily!) along the way

 

UDP

iperf -c <ip> -u------------------------------------------------------------Client connecting to <IP>, UDP port 5001Sending 1470 byte datagramsUDP buffer size:  208 KByte (default)------------------------------------------------------------[  3] local <ip> port 44731 connected with <ip> port 5001[ ID] Interval       Transfer     Bandwidth[  3]  0.0-10.0 sec  1.25 MBytes  1.05 Mbits/sec[  3] Sent 893 datagrams[  3] Server Report:[  3]  0.0-10.0 sec  1.25 MBytes  1.05 Mbits/sec   0.035 ms    0/  893 (0%)

The same pair of hosts using TCP

iperf -c <IP>------------------------------------------------------------Client connecting to <IP>, TCP port 5001TCP window size: 85.0 KByte (default)------------------------------------------------------------[  3] local <IP> port 50024 connected with <IP> port 5001[ ID] Interval       Transfer     Bandwidth[  3]  0.0-10.0 sec   112 MBytes  93.8 Mbits/sec

 So my first question is:

 

Is there a hidden option to force the sender to use TCP (or the receiver to request TCP connections from senders?), or is it at the moment only available in custom test builds?

 

I think it is of general interest, because in this case it is a limitation of the network the receiving server is in (I got the same iperf results sending from two different networks) . It is hosted in an OVH datacenter (a french provider, pretty big in the EU), and I guess it might be related to their DDoS protection stuff. However, this is just a lucky guess as this is my first OVH hosted host I am not very familiar with their network.

 

Anyway, I suppose there will be quite a few people renting el-cheapo servers as secondary offsite backup, that might run into this problem. I hope my post will make it at least googleable.  But can almighty RomanZ maybe even solve it :D ?

Share this post


Link to post
Share on other sites

Ok, it turns out I do not know how tu use iperf... UDP requires to set a sending rate. As in 

iperf -c <IP> -u -b 20m

if you want to check 20 MBit. Turns out the host is still slower in UDP: Around 20-25 MBit, afterwards the dropped packet rate skyrockets. However, this does not explain BTSyncs << 10Mbit speeds. 

 

As for now I am trying with a TCP-OpenVPN that tunnels BTsync traffic directly to the machine. However as the Mac cleint exploded on me, and after crashing restarted indexing (and even asked a new identity and restarted the 30 day trial), I can not yet say what the results will be.

 

For the time being the receiver still throws millions of

[20150316 15:55:27.718] 10.8.0.6:59696: did not pick any blocks. blocking peer temporarily[20150316 15:55:57.593] Disconnect: Is seed[20150316 15:55:41.725] Disconnect: Timeout due to inactivity

into the log

Share this post


Link to post
Share on other sites

I get the feeling this is a limitation of BTSync not being able to sync large amount of files. The best I manage to do with a medium sized share (16GB, 16k Files, various sizes) was around 2 MB/sec, which will occasionally drop to less then 1 MB/sec in the "sawtooth" pattern described here. No matter it is local or remote or TCP or UDP.

 

However when I add the share I originally want to sync (400GB, 420k Files, Most 8MB), it never goes over 1 MB/sec. Also, when the large share is added to BTsync, even when syncing is paused, the medium sized share mentioned above will also not go over 1 MB/sec.

 

I am not sure I am CPU bound. One machine is an oldish Core 2 DualCore, the other a Dual Core (4T) Cedarview Atom. While BTsync has significant impact on CPU, the CPU usage for both machines is not nearly maxed out under the conditions lined above.

 

Is there something in BTSyncs architecture that prevents it from syncing large collections? Some event-processing/thread locking limitation? Some scaling problem with a large number of files? Did anybody ever saw BTSync achieving or surpassing 100MBit/s when the files are large enough?

 

Also I did notice some largish sqlite3 database (536 MB) created by btsync. And it seems to have no indices. However, as I do not know whether they are read or written more often I do not know whether this could be a performance problem. Just worked a lot with sqlite3 before, and noticed that it can go from awesome! to yuck! pretty quickly once there is a lot data in it....

 

I did try the "speed profile" thing for a few minutes, but it just collected 10 MB of data I can not interpret by myself :)

Share this post


Link to post
Share on other sites

@sebijisi

Indeed, there is some issue in processing large amount of small files by Sync - and they can affect data transfer speed significantly. The sqlite DB is not a bottleneck here. We plan to optimize it in future.

 

For the TCP connections instead of UDP ones - there is a debug flag which you can set to force Sync to use TCP to connect. Although note, that UDP is much more flexible in establishing direct connection, so forcing TCP may result in inability to directly connect in some cases.

 

So, to force TCP-only

- create a debug.txt file in storage folder (see this article for possible locations depending on OS)

- put the value of "00000000<newline>00000002" into this file

- save file, restart Sync. Do on all peers involved.

 

Disclaimer (to everyone who did not read the warning above)

Use with care. May cause inability to connect peers directly.

 

Let me know if it helped to improve your syncing speed.

Share this post


Link to post
Share on other sites

I've created an account to try and solve this issue.  I've followed your instructions, but I'm not sure I've done everything correctly.

 

I've found the debug.txt file on both machines.  There is only one line of text inside that reads "FFFFFFFF0".

 

Do I delete this line, or leave it where it is?

 

Also, your instructions say, put the value of "00000000<newline>00000002" into this file.

 

Does that mean like this:

 

00000000

00000002

 

?

 

I've tried this, but I'm not sure how to check if it's forcing TCP.  It doesn't seem to have fixed the problem.

 

Thanks in advance.

Share this post


Link to post
Share on other sites

@Herdo

You can leave the first line as is. It is only responsible for what to log and what not to log. Setting it to zero lets Sync know that you no longer want to write logs. Indeed, debug.txt should consist of 2 lines:

00000000 (or FFFFFFFF)00000002

The second line turns off UDP traffic, which in certain cases causes traffic slow down. Although, I would recommend first disabling the "hdd_low_priority" advanced setting. In majority of cases HDD is the bottleneck.

 

Also, could you please share a bit more details about your case? Are peers connected via LAN or internet? What are the peers OS and hardware? Channels bandwidth?

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.