Lots Of Currentcontrolset\control\cryptography Reads When Loading

[himselfv]

Some of my shares have tens of thousands files, and I've noticed they're loading very slow. When starting Sync, even before it starts "(Indexing...)" the folder, it is stuck for a whole minute simply adding it to "My Sync" folder list.

 

I fired up SysInternals Process Monitor, and it seems there's nothing but this set of requests again and again:

 

12678438    11:36:24,3138382    BTSync.exe    3476    RegOpenKey    HKLM\System\CurrentControlSet\Control\Cryptography\Providers    REPARSE    Desired Access: Read
12678439    11:36:24,3138513    BTSync.exe    3476    RegOpenKey    HKLM\System\CurrentControlSet\Control\Cryptography\Providers    SUCCESS    Desired Access: Read
12678440    11:36:24,3138673    BTSync.exe    3476    RegCloseKey    HKLM\System\CurrentControlSet\Control\Cryptography\Providers    SUCCESS    
12678441    11:36:24,3138791    BTSync.exe    3476    RegOpenKey    HKLM\System\CurrentControlSet\Control\Cryptography\Configuration    REPARSE    Desired Access: Read
12678442    11:36:24,3138915    BTSync.exe    3476    RegOpenKey    HKLM\System\CurrentControlSet\Control\Cryptography\Configuration    SUCCESS    Desired Access: Read
12678443    11:36:24,3139053    BTSync.exe    3476    RegCloseKey    HKLM\System\CurrentControlSet\Control\Cryptography\Configuration    SUCCESS    
 

These 6 requests are repeated a godless number of times, probably the same tens of thousands times, once for each file. There's no reading apart from rare block reads from btsync ".db" files, no network activity, justs these requests. This takes minutes.

 

I'm guessing here, so pardon me if I'm wrong, but perhaps it's something that can be easily optimized? Maybe you're creating some sort of crypto primitive which looks into these keys on initialization, and in fact you could use the same primitive for all files? Because the actual "(Indexing...)" step runs much, much faster than this step.

[RomanZ]

@himselfv

 

I suspect that the things you see in the ProcMon is rather the symptom than an issue. We do not  read \ write the keys you mention directly, but we use Microsoft CryptoAPI which does. 

 

Couple of things to clarify and to try:

- Are you using Encrypted folders in your setup?

- If you are syncing in LAN - you may try to disable encryption in LAN.

- How many files and folders in general do you sync?

- Does your hardware match the amount of files and folders (compare your RAM size to the amount of RAM consumed by Sync).

[himselfv]

> Are you using Encrypted folders in your setup?

 

Yes, and the folder in question is of encryptable kind (decrypted on this PC though).

 

> If you are syncing in LAN - you may try to disable encryption in LAN.

 

No, syncing over the internet.

 

> How many files and folders in general do you sync?

 

5 sync nodes: 2300 files, 50 files, 8000 files (the problematic one), 2000 files, 1300 files. Only the problematic one is encryption-enabled, others appear instanteniously and start "(Indexing...)".

 

> Does your hardware match the amount of files and folders (compare your RAM size to the amount of RAM consumed by Sync).

 

Probably. 8 Gb RAM, virtual memory allocated by BTSync when fully loaded: 226 Mb.

 

 

 

> We do not  read \ write the keys you mention directly, but we use Microsoft CryptoAPI which does.

 

Yeah, I thought it's something like that, but maybe you're instantiating some kind of CryptoAPI object for each file where one would have been enough?

 

I'm just guessing, it just looks like this slowness happens when BTSync is reading *.db files (probably SQLite databases?), and the largest of those on my PC is 94 Mb. So disk-wise it could be read in an instant.

 

I've run the API Monitor, and it seems BTSync does this for every file (e.g. 8000 times):

 

BCryptOpenAlgorithmProvider(... 'AES')

BCryptGetProperty('ObjectLength')

BCryptSetProperty('ChainingMode')

BCryptImportKey(...'KeyDataBlob')

BCryptEncrypt(...)

BCryptEncrypt(...)

BCryptCloseAlgortihmProvider()

BCryptDestroyKey()

 

MSDN says:

 

> Because of the number and type of operations that are required to find, load, and initialize an algorithm provider, the BCryptOpenAlgorithmProvider function is a relatively time intensive function. Because of this, we recommend that you cache any algorithm provider handles that you will use more than once, rather than opening and closing the algorithm providers over and over.

 

Maybe this is the reason?

[RomanZ]

@himselfv

 

Thanks for the comprehensive analysis. We'll check if this is true indeed and if it can be optimized.

[RomanZ]

@himselfv

 

Indeed, CryptoAPI calls could be optimized for encrypted peers. We'll implement it for upcoming 1.4 release.

 

Thanks a ton for very technical issue analysis, we really appreciate that!