High Memory And Cpu Utilisation On Windows 7

[Anro]

Services.exe process utilize more than 8Gb of ram and the whole system becomes less responsive when folder rescan occurs.

I've set 300 seconds folder rescan interval just to demonstrate correlation between spikes on "services.exe" stats and rescanning (x scale covers about 15 minutes):

 

 

Synced folders are encrypted with Windows EFS.

Edited December 16, 2014 by Anro

[GreatMarko]

Can you share the number/size of files/folders being monitored by your Sync?

 

Some factors that can increase the amount of memory and/or CPU usage that Sync uses:

• Using encrypted folders
• Monitoring a large number of files/folders
• Having "disk_low_priority" set to false
• Having "lan_encrypt_data" set to true
• Having "recv_buf_size" and "send_buf_size" set very high
• Having "folder_rescan_interval" set very low

Given that your "spikes" are occurring when Sync performs folder rescans, they're probably caused by a very large amount of files/folders being indexed, and/or the fact that you're using encrypted folders.

[Anro]

There are 2 synced folders:

1) 66 456 Files, 4 217 Folders

2) 2 437 Files, 419 Folders

 

Disabling encryption on synced folders workarounded the issue. But this is not an option for me.

Edited December 16, 2014 by Anro

[Anro]

Given that your "spikes" are occurring when Sync performs folder rescans, they're probably caused by a very large amount of files/folders being indexed, and/or the fact that you're using encrypted folders.

I don't agree that rescanning even hundred thousands of files should lead to consuming almost all available RAM.

I performed simple random reading tests in encryted folders and I could not reproduce condition where services.exe ate too much memory. Looks like btsync reads files another way comparing to my silly tests. Maybe btsync scanning could be slowered somehow?

[GreatMarko]

Well, I think you've already answered your question to an extent; Disabling encryption on synced folders workarounded the issue.

Whilst I'm no expert on Windows EFS, what is the size of the largest file in one of the encrypted folders you're trying to sync? It's not around 8GB per chance? Given that it's "services.exe" that's generating the CPU/memory load, maybe the issue is due to the Windows sub-system having to perform decryption before the files can be accessed by Sync?

[Anro]

I performed some testing and it looks like this issue appears only when there are many small files/directories.

Big files does not cause such behavior.

 

Windows EFS is definitely have to decrypt files on access, but I could not reproduce the issue with other than btsync processes, reading files in encrypted dir.

 

Could you suggest some testing tool that can generate similar to btsync IO activity in specified folder?

[RomanZ]

@Anro

Sync does not do anything special with files upon rescan. It goes over all files in file tree and checks every file mtime and size. If any of these changed - it also reads and hashes file's content. I strongly suspect that Win implementation of EFS tries to optimize access to file content and decrypts content into memory just in case if app is going to access it.

 

You can write a simple batch file (or any other script / programming language code) accessing all the files and see if you'll experience same issue as with Sync.