[Very Frequent Indexing, High Cpu And Battery Usage With V2.3 On Mac]

2 minutes ago, Helen said:

to rescan the folder periodically and detect the changes in files. system notifications sometimes fail.

Have you had reports of system notifications failing on macOS or Windows?

[SQLite WAL files gouging disk on peers]

Yes, they disappear on exit. Nonetheless the idea behind Sync is to keep it running all the time...

[Very Frequent Indexing, High Cpu And Battery Usage With V2.3 On Mac]

Just now, Helen said:

I was saying it needs rescan. so I cannot answer "why it doesn't need it"

Any idea why "it (Sync) needs a rescan" on macOS?

[Very Frequent Indexing, High Cpu And Battery Usage With V2.3 On Mac]

@Helen, that's interesting -- any idea why Sync doesn't need a rescan on macOS? Thanks for the link to the debug log collection.

[very big ram footprint]

@operationally Linux is very different to macOS in that there is very little filesystem overhead to monitor large number of files in a directory tree: you don't have to have an inotify object for each file/folder being monitored. In this case, the 1.31GB overhead is almost completely application-specific memory overhead, which seems excessive to me.

[very big ram footprint]

I'm seeing massive RAM usage on macOS.  On a MacBook 12" with 8GB of RAM, I'm seeing Resilio Sync use 1.31GB of memory for a two-folder share with 1,654,405 files.  That's aside from the massive disk usage for the SQLite files.

I'm sure there is a lot of room for improvement here.  I'm debating whether Resilio Sync is worth the overhead compared to a periodic rsync in my case.

I wonder if some of the simple suggestions on StackOverflow for limiting SQLite memory usage could be helpful to the devs.

[SQLite WAL files gouging disk on peers]

In ~/Library/Application Support/Resilio Sync on macOS I can see the SQLite databases being used to track shared folders.

The WAL files are massive!  In my case, for a share of two folders (having 1,654,405 files in many sub-folders) the entire folder is 2.86GB on each peer.  About 1.1GB of that are wasted in WAL files while Resilio Sync is idle.

I can appreciate the need for using SQLite to track files, but surely these WAL files can be periodically cleaned whilst Sync is idle?  Or at least an effort to limit the maximum WAL size.

[Very Frequent Indexing, High Cpu And Battery Usage With V2.3 On Mac]

I also had high CPU utilization on two macOS machines, one being a MacBook 12" (2017) which was overwhelmed by Resilio Sync.

I am managing just two folders, but each has 1,654,405 files combined, over many sub-folders.  Sync was not transferring files despite the high CPU: all files were in a synced state.

It turns out that Sync was periodically re-scanning all of the files/folders every 10 minutes!  After disabling this, using the "power user preferences" option "folder_rescan_interval = 0", the high CPU usage disappeared completely on both machines.

Sync was still able to detect changes to each folder.  I suspect they are using the macOS FSEvents APIs where the OS tells Sync which files have changed without polling.

I would love to know why the folder_rescan_interval isn't set to 0 on macOS by default?  (And Windows, where I'm guessing the equivalent Windows Change Journals are being used?).