joepc Posted September 19, 2013 Report Share Posted September 19, 2013 Hi, I have been trying to sync a windows directory to another server for a few days now and all the usual file comparison / sync apps fail due to the number of files. I set-up BitTorrent sync and it appears to be working great but I'm a little concerned the default settings might not be ideal for this data? Does anyone have experience of syncing a large amount of small files? LAN to LAN Thanks, Joe Quote Link to comment Share on other sites More sharing options...
sushi17 Posted September 20, 2013 Report Share Posted September 20, 2013 I have sync'd 2 shares for a total of roughly 400gb (50,000 files), default settings worked fine. This was 1 site pushing data to 2 sites over internet, took a while, LAN should be much faster. Quote Link to comment Share on other sites More sharing options...
deem Posted September 20, 2013 Report Share Posted September 20, 2013 From unofficial FAQ we know that "if you're monitoring/syncing 1 million files, you will need around 300-400 MB of free memory". BTSync is 32-bit, so, it can only use ~2 GB of RAM. Therefore, the maximum amount of files is limited to ~6 million. Quote Link to comment Share on other sites More sharing options...
nils Posted September 20, 2013 Report Share Posted September 20, 2013 Hi Joe, Regarding the default settings, you could turn off DHT, tracker and relay server and turn on search LAN, if you only intend to use it within a LAN, maybe even insert your hosts in the predefined hosts section. @deem: I use the x64 version of btsync and this the output of file I get on the btsync executable, so I think BTsync is 64 bit capable on supported platforms, thereby expanding the maximum number.btsync: ELF 64-bit LSB executable, x86-64, version 1 (FreeBSD), dynamically linked (uses shared libs), stripped Quote Link to comment Share on other sites More sharing options...
mbob Posted September 20, 2013 Report Share Posted September 20, 2013 I have very good experiences for large volume sycncing with "Syncovery" (http://www.syncovery.com/) Hi, I have been trying to sync a windows directory to another server for a few days now and all the usual file comparison / sync apps fail due to the number of files. I set-up BitTorrent sync and it appears to be working great but I'm a little concerned the default settings might not be ideal for this data? Does anyone have experience of syncing a large amount of small files? LAN to LAN Thanks, Joe Quote Link to comment Share on other sites More sharing options...
deem Posted September 20, 2013 Report Share Posted September 20, 2013 @nils: Ok, BTSync for windows is 32-bit and the author said: "sync a windows directory". Make conclusions. The author can dance with other settings, but it won't help. Problem is solved. You can't sync so many files on 32-bit app. Quote Link to comment Share on other sites More sharing options...
joepc Posted September 20, 2013 Author Report Share Posted September 20, 2013 Thanks for the comments, after 50 hours it synced to 800K (90GB) files and slowed to a crawl - the process was consuming nearly 1GB ram so deem is correct. I will try Syncovery. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.