Controlling RAM usage


MRDowd

Recommended Posts

Referencing Helen's explanation in the very big ram footprint thread:

Quote

And still I'll explain it, which may help to find a solution. This has been already explained multiple times on forum, but here it goes. 
RAM is taken by database files. Database keeps record of each entry in the sync share - currently present files and the ones that have ever been in the share. Entry - is the file itself, and also each subdir level in its path. Each entry takes up around 2KB of RAM. The bigger folder tree you have, the more nested folders, the more file changes/deletions, the bigger DB grows, as Sync needs to remember all those entries. 

While this explains what problems exist, checking through a few pages of search results didn't show me any methods to actually tame memory usage.

Currently, my Resilio process is using 2.6 GB of RAM memory. On an 8 GB machine that needs to run other applications at the same time (especially Microsoft's bloated Office suite), this is unacceptable. For a long time my RAM usage has constantly been high 80% to low 90%, which causes slowdowns once stuff starts hitting the page file.

My team at work uses Resilio to sync 3 folders (all of us are owners):

#1 has a small number of large files (3,000 files + subdirs, 276 GB size)

#2 has a more moderate number of smaller filed (5,000 files + subdirs, 75 GB size)

#3 has a large number of much smaller files (170,000 files + subdirs, 100 GB size)

I've seen a few posts on this forum of people synchronizing much more stuff than this with significantly less RAM usage. Since I don't really know how much RAM each folder is using, I've been reduced to speculating.

The files in #1 are very static. After they are put in the folder, they never change and are only removed when we don't need them anymore. #2 is roughly the same. The changes are more frequent, but not excessively so. It's very likely that these two folders are not a problem. I believe #3 is the culprit for the vast majority of our RAM usage. Folder #3 is my team's working directory, and a normal day can involve each of the 3 team members modifying/creating/deleting several hundreds or thousands of files that our programs generate. We will also archive and remove old content to save space and reduce file count, which causes further modifications.

Our workflow was created when we were using Dropbox, which we had to migrate away from. DB had its issues, but it was nowhere near the memory hog that Resilio is right now. Significantly changing out workflow to optimize it for Resilio's quirks is likely to be quite disruptive to our daily activities. With that in mind, are there ways to tame the excessive RAM usage that I believe is caused by the large amount of file changes? I took a look through the power user settings, but nothing jumped out at me as useful for this problem. In one post I saw a recommendation to disconnect and reconnect the folder. Is this safe to do on a folder that might still be syncing changes?

Link to comment
Share on other sites

  • 2 weeks later...
35 minutes ago, Helen said:

Go to sync storage folder and see the size of .db files. 

There's no  option to limit RAM for Sync in sync settings. To reduce RAM consumption you need to remove the share from Sync so that db gets cleared. 

The db files are gigantic. One is 4.7 GB, a second is 2.3 GB. The entire folder (minus the backup folder) is 8.25 GB.

What exactly must I do to purge the database? Is just disconnecting+reconnecting fine? Or do I need to disconnect+remove the folder, and get a new link from my coworker? Is this procedure safe to do on a folder that has not been fully synced yet?

This RAM issue is a sticking point for us. We're already running on SSDs so hitting the DB on the drive wouldn't hurt us as much as other people with platter drives. We could consider upgrading out RAM, but if it's going to keep ballooning out of control like this that doesn't seem like a long-term option.

screenshot.209.png

Link to comment
Share on other sites

19 hours ago, MRDowd said:

Or do I need to disconnect+remove the folder, and get a new link from my coworker? Is this procedure safe to do on a folder that has not been fully synced yet?

 you need to remove (disconnect+remove) the share from Sync. Do that on all peers and re-share the new link\key. 

Simply disconnecting the folder won't delete the database. 

Link to comment
Share on other sites

  • 3 months later...

I am just having the same issues with RAM usage on all the machines sharing my files.

But because I am sharing around 20 folders, this is not an option, if I delete them I have to get to all this devices (on multiple locations) and add the link again, couldn't you just introduce a "compress" db or "clean" db option ?

I mean you are also addressing Enterprise, so I wouldn't wonder if they complain.

BTW I am a paying customer as well.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.