[Win] Huge 6Tb+ 300K+ Files Folder, Ad And Dfs - Consideraions

[dvlad666]

Dear devs and community!

 

First, let me thank everybody for supporting the development of such a great client!

 

I want to deploy BT Sync as an addition to syncing HUGE 6TB+ 300k+ files folder between domain servers running DFS Replication and VSS, and non-domain workstations and would like to hear if there are any considerations.

 

My current scenario is the following:

1. There are 3 physical sites with 3 Windows Server 2003R2 32bit servers running in single domain (all three are domain controllers)

2. This huge folder is being replicated using DFS-R (DFS Replication) between all 3 servers, located on some NTFS volumes and backuped inside that volumes with Volume Shadow Copy mechanism (last 61 snapshots) daily

3. One server is considered as "Main" as it is big, physical and 24|7 online. The second one is movable from apartment to apartment with me and actually "hosts the folder" for the workstation i work on

The third one is a virtual machine to work truly on the go when i need that folder with me

3. All workstations connect to the servers via SMB (network file sharing) to access the folder

 

My desired scenario using BT-Sync:

1. Main server hosts both DFS-R and BT-Sync replication over the same folder

2. Second server remains with DFS-R and shares folder for workstations via SMB

3. Third server is eliminated (was used as a VM for mobile workstations)

4. Truly mobile workstations (Win XP, Win 7 32|64)  have full copy of the folder synced via BT-Sync

 

Thus no duplication between BT Sync and DFS should exist as only the main server will have both, all other nodes will use only one or another

 

I've searched the forum and came up empty, my basic questions are the following:

1. Can BT Sync live together with DFS-R? Both syncs should be fully bi-directional

2. Can BT Sync handle such a huge amount of data? (DFS-R does never rescan the whole folder. it uses USN, AD and some other mechanisms to sync only changes)

3. Can BT sync live together with VSS? I.e.:

3.1 Does it work well during the process when the Shadow Copy of the volume is being created?

3.2 Can it break down VSS capability on the volume as, say, some Acronis products do?

4. When i deploy BT Sync, what is the best way to pointing BT Sync to the "current" big folder on the master server and point the client to the "new" place where the folder's copy should be to make sure that: 

4.1 Big folder contents do not get erased with the "empty" folder from the workstation

4.2 Workstation gets full copy of the big folder and then only changes are synced

 

Thanks a lot for your help!

 

Best,

Vlad

[nils]

Hello Vlad,

 

1. As long as DFS-R is not locking files permanently, BTsync should be able to run just fine.

2. I have read about other users with syncs going into multiple TB, so no problem there. 300+k files seems reasonable, only remark would be to have enough RAM.

3. No idea, I do not use VSS or any Windows Server.

4.1. Set up the master server first: create a new share, point it to the current folder and wait until indexing is done.

4.2. Add the RW key to your workstations, select an empty folder and it should receive the data from your master server.

 

HTH

[fukawi2]

I think your biggest issue will be that btsync won't maintain any permissions across the sync'ed folder(s).

[syadnom]

I've synced up to 1.3TB with btsync (even pre-seeded the target) and it works fine, though I haven't done such a large amount of files.

 

file permissions aren't going to come over for sure...you might have to backup the folder structure and permissions and transfer those first.  not too big of a deal if your folders all inherit permissions but if you block inheritance or do anything advanced then permissions may be an issue.