MRDowd Posted August 23, 2017 Report Share Posted August 23, 2017 Referencing Helen's explanation in the very big ram footprint thread: Quote And still I'll explain it, which may help to find a solution. This has been already explained multiple times on forum, but here it goes. RAM is taken by database files. Database keeps record of each entry in the sync share - currently present files and the ones that have ever been in the share. Entry - is the file itself, and also each subdir level in its path. Each entry takes up around 2KB of RAM. The bigger folder tree you have, the more nested folders, the more file changes/deletions, the bigger DB grows, as Sync needs to remember all those entries. While this explains what problems exist, checking through a few pages of search results didn't show me any methods to actually tame memory usage. Currently, my Resilio process is using 2.6 GB of RAM memory. On an 8 GB machine that needs to run other applications at the same time (especially Microsoft's bloated Office suite), this is unacceptable. For a long time my RAM usage has constantly been high 80% to low 90%, which causes slowdowns once stuff starts hitting the page file. My team at work uses Resilio to sync 3 folders (all of us are owners): #1 has a small number of large files (3,000 files + subdirs, 276 GB size) #2 has a more moderate number of smaller filed (5,000 files + subdirs, 75 GB size) #3 has a large number of much smaller files (170,000 files + subdirs, 100 GB size) I've seen a few posts on this forum of people synchronizing much more stuff than this with significantly less RAM usage. Since I don't really know how much RAM each folder is using, I've been reduced to speculating. The files in #1 are very static. After they are put in the folder, they never change and are only removed when we don't need them anymore. #2 is roughly the same. The changes are more frequent, but not excessively so. It's very likely that these two folders are not a problem. I believe #3 is the culprit for the vast majority of our RAM usage. Folder #3 is my team's working directory, and a normal day can involve each of the 3 team members modifying/creating/deleting several hundreds or thousands of files that our programs generate. We will also archive and remove old content to save space and reduce file count, which causes further modifications. Our workflow was created when we were using Dropbox, which we had to migrate away from. DB had its issues, but it was nowhere near the memory hog that Resilio is right now. Significantly changing out workflow to optimize it for Resilio's quirks is likely to be quite disruptive to our daily activities. With that in mind, are there ways to tame the excessive RAM usage that I believe is caused by the large amount of file changes? I took a look through the power user settings, but nothing jumped out at me as useful for this problem. In one post I saw a recommendation to disconnect and reconnect the folder. Is this safe to do on a folder that might still be syncing changes? Quote Link to comment Share on other sites More sharing options...
Helen Posted September 5, 2017 Report Share Posted September 5, 2017 Go to sync storage folder and see the size of .db files. There's no option to limit RAM for Sync in sync settings. To reduce RAM consumption you need to remove the share from Sync so that db gets cleared. Quote Link to comment Share on other sites More sharing options...
MRDowd Posted September 5, 2017 Author Report Share Posted September 5, 2017 35 minutes ago, Helen said: Go to sync storage folder and see the size of .db files. There's no option to limit RAM for Sync in sync settings. To reduce RAM consumption you need to remove the share from Sync so that db gets cleared. The db files are gigantic. One is 4.7 GB, a second is 2.3 GB. The entire folder (minus the backup folder) is 8.25 GB. What exactly must I do to purge the database? Is just disconnecting+reconnecting fine? Or do I need to disconnect+remove the folder, and get a new link from my coworker? Is this procedure safe to do on a folder that has not been fully synced yet? This RAM issue is a sticking point for us. We're already running on SSDs so hitting the DB on the drive wouldn't hurt us as much as other people with platter drives. We could consider upgrading out RAM, but if it's going to keep ballooning out of control like this that doesn't seem like a long-term option. Quote Link to comment Share on other sites More sharing options...
uncovery Posted September 5, 2017 Report Share Posted September 5, 2017 There should be a process that allows people to erase the file history of a folder and make all connected machines assume that the folder is "new". Is a disconnect achieving that? Because then at least I could periodically dis-and reconnect a folder so reduce the RAM usage. Quote Link to comment Share on other sites More sharing options...
Helen Posted September 6, 2017 Report Share Posted September 6, 2017 19 hours ago, MRDowd said: Or do I need to disconnect+remove the folder, and get a new link from my coworker? Is this procedure safe to do on a folder that has not been fully synced yet? you need to remove (disconnect+remove) the share from Sync. Do that on all peers and re-share the new link\key. Simply disconnecting the folder won't delete the database. Quote Link to comment Share on other sites More sharing options...
Guest Posted December 18, 2017 Report Share Posted December 18, 2017 I am just having the same issues with RAM usage on all the machines sharing my files. But because I am sharing around 20 folders, this is not an option, if I delete them I have to get to all this devices (on multiple locations) and add the link again, couldn't you just introduce a "compress" db or "clean" db option ? I mean you are also addressing Enterprise, so I wouldn't wonder if they complain. BTW I am a paying customer as well. Quote Link to comment Share on other sites More sharing options...
Helen Posted December 19, 2017 Report Share Posted December 19, 2017 Quit Sync, go to storage folder, delete all .db files and all peers. It's not a designed way though, so better not use that frequently. But seems easier that reconnecting 20 folders. And yes, Sync will still need to reindex all the files build new database and merge folder trees. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.