Syncing large number of files?

[andy987s]

We have ~160 different project folders, some of which contain 1 million files. It's about 5.5T in total.

These are being synced to half a dozen different sites around the globe (using Unison) ATM.

I'm wondering if BTSync can handle this efficiently:

• When a file changes - is there a filesytem watcher that can trigger it to just re-index the changed file. Or will it scan the metadata for every file at a set interval only.
  
• Could we write our own filesystem watcher that hooks into the Filesystem API, and then queues changed files to BTSync rather than have it traverse the whole directory tree frequently (as that is an expensive operation in our case). Then schedule full scan weekly.
  
• If we are exceeding the limit for the number of files to sync, can we run multiple BYSync processes on the same host (on different ports) in order to split the filesystem up into manageable chunks?
  

[GreatMarko]

• When a file changes - is there a filesytem watcher that can trigger it to just re-index the changed file. Or will it scan the metadata for every file at a set interval only.
  

Files are monitored in real-time. In addition, as a "fallback", folders are additionally re-scanned every 10 minutes (this interval is configurable)

• Could we write our own filesystem watcher that hooks into the Filesystem API, and then queues changed files to BTSync rather than have it traverse the whole directory tree frequently (as that is an expensive operation in our case). Then schedule full scan weekly.
  

At present it's not possible to "schedule" scans directly with BitTorrent Sync. I suppose the workaround would be to create scheduled tasks that would run BTSync.exe at specific times/kill it at others.

• If we are exceeding the limit for the number of files to sync, can we run multiple BYSync processes on the same host (on different ports) in order to split the filesystem up into manageable chunks?
  

There are no limits imposed by BitTorrent Sync on the number of files you can sync.

In relation to running multiple instances, one of the developers has previously commented "If you configure different ports I think it should. But I haven't tried this combination."