Memory Footprint On Large Data Sets?


Recommended Posts

Hello All

 

I copied all the Files of one file server to a brand New ReadyNAS 104 (about 3 Terabytes of files). After that I tried several times to let it finish sync (it only compares the Files, as I see it, there are no data copied). It always starts very fast, but after a while it takes forever to sync as more and more files are inside the dataset.. any thoughts on that? The idea is that the ReadyNAS creates a exact Copy of my Business Files at home, so I have an automated Offside Backup. 

 

The ReadyNAS has only 512 MB RAM. I suspect, that it cannot cope with a Database for 120'000 Files in 3 Terabyte of date.. could that be the reason? Any other ideas?

 

regards, Urs

Link to comment
Share on other sites

@dsolvegmbh

 

BTSync consumes around 400 bytes of memory per file(folder). So, 512 Mb RAM should be enough to track all the 120k files. You can try to run

top -p <PID_of_BTSYNC>

command to see how much memory (in percent of all available system memory) it consumes - and get some feeling if it has enough memory.

 

Also, are you using 1.3.106 build? There was important fix related to the indexing of files.

Link to comment
Share on other sites

The Problem is that NetGear has not released an update to 1.3.106 ... so I am stuck with what they have on their App Store  :-(

 

On one Computer that is syncing the Memory Footprint is About 216 MB .. but it has more folders to sync. But still, for some reason things get very slow, the more files there are..

Link to comment
Share on other sites

@dsolvegmbh

We are working with Netgear to deliver the latest version to the package. Also, you can replace the binary yourself if you have the SSH access to the box.

I would also suggest trying to increase folder_rescan_interval to 1 day as a test (due to numerous files it looks like it takes a lot of time to rescan folders). Though, it will require SSH access as well - config mode requires you to run binary with "--config" switch.

Link to comment
Share on other sites

  • 3 weeks later...

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.