cloudbitsync Posted March 19, 2014 Report Share Posted March 19, 2014 Hi there, We're doing some testing with sharing lots of small files. I made a batch file to create 200,000 1MB text files as below: @echo offclsSet Sleep=0:startif %Sleep% == 200000 ( goto end )fsutil file createnew c:\bitsyncbreaktest\%Sleep%.txt 1048576Set /A Sleep+=1goto start:endecho "am 200000 now"pauseWhen syncing to our server, each file only gets up to around 30-50 kilobytes/sec. I created some random 1GB files and those sync at 12MB/sec. Why do small files sync so slowly, is this by design? Cheers. Quote Link to comment Share on other sites More sharing options...
TheDurtch Posted March 19, 2014 Report Share Posted March 19, 2014 I am thinking with the small files it has to start syncing every single file it is never able to reach full speed. At least that is true when copying files. Quote Link to comment Share on other sites More sharing options...
RomanZ Posted March 19, 2014 Report Share Posted March 19, 2014 Why do small files sync so slowly, is this by design? BTSync syncs file one by one, not a bulk transfer. However, it should be done faster - looks like some issue. I'd like to find out where is a bottleneck. Could you please conduct couple of tests? 1) Check the CPU / memory usage when you see such a slow speed.2) Try making the same test with only ~1000 files. Thanks! Quote Link to comment Share on other sites More sharing options...
cloudbitsync Posted March 20, 2014 Author Report Share Posted March 20, 2014 Thanks for your reply, we can't identify the bottleneck. The server load is also 0.01 With 200000 files: With 1000 files: I should add, it was very slightly faster with 1000 files, but not by much. Quote Link to comment Share on other sites More sharing options...
RomanZ Posted March 20, 2014 Report Share Posted March 20, 2014 cloudbitsync, I can propose 2 more things to try. 1) In advanced preferences set "disk_low_priority" to False.2) If your servers are in LAN - try enabling "lan_use_tcp" in advanced options. Quote Link to comment Share on other sites More sharing options...
cloudbitsync Posted March 21, 2014 Author Report Share Posted March 21, 2014 1 . Yep, I set that when we first set it up, didn't really make a difference 2. We're doing this over the internet Any other ideas? Cheers Quote Link to comment Share on other sites More sharing options...
cloudbitsync Posted March 25, 2014 Author Report Share Posted March 25, 2014 Bump Any other ideas? Quote Link to comment Share on other sites More sharing options...
RomanZ Posted March 25, 2014 Report Share Posted March 25, 2014 cloudbitsync, Yes. Wait a little bit, we are going to release 1.3 soon, and there is a good chance it will resolve your issue. Quote Link to comment Share on other sites More sharing options...
cloudbitsync Posted March 25, 2014 Author Report Share Posted March 25, 2014 Awesome, cheers RomanZ Quote Link to comment Share on other sites More sharing options...
cloudbitsync Posted March 26, 2014 Author Report Share Posted March 26, 2014 cloudbitsync, Yes. Wait a little bit, we are going to release 1.3 soon, and there is a good chance it will resolve your issue. This release has certainly improved speeds, however it seems that BTSync incorrectly reports speeds within the client, both on Linux and Windows. We are now getting between 2-4MB/s and the client reports 0-400kB/s Quote Link to comment Share on other sites More sharing options...
RomanZ Posted March 26, 2014 Report Share Posted March 26, 2014 cloudbitsync, Thanks for reporting, we'll take a look why is it happening. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.