Syncing Large Folder Structure

[second_coming]

Hi all,

just recently started using BTSync for a few things at home at work and have to say I'm loving it

I've come up against an issue when trying to sync the contents of MDaemon mail server (running on Windows 2008 R2 foundation server).

The entire folder for MDaemon including all the user mailboxes etc comes to 114.6 Gb and 671465 files.

Prior to version 1.1.15 BTSync was forcing the server to go OOM. Since updating to 1.1.15 BTSync now seems to be much better optimised and scans the folder correctly (obviously which takes a long time).

I have 2 clients trying to sync with the server but they are not copying the data across and when you look on the 'devices' tab they are not reporting the correct amount of data to download (one says 23.8 Gb and the other is showing 6.9Mb).

I don't know whether the amount of files is an issue?

Any ideas as to how to get this working properly?

thanks

[GreatMarko]

Some things to check:

1) Are you running the same version of BitTorrent Sync on all devices?

2) Are the corresponding .SyncIgnore files identical on all devices?

3) Does your mailserver keep any files you're trying to sync open/locked? - if so, it may be worth excluding these from syncing

4) With BitTorrent Sync not running on any of your devices, locate any .!Sync files (incomplete transfers), manually remove them and then restart Sync on all your devices.

[second_coming]

Hi GreatMarko

1) I am running 1.1.15 on all the systems

2) I haven't set anything to be ignored in the .SyncIgnore files

3) possibly, I'll have to check

4) I'm currently in the process of clearing the BitTorrent Sync folder in AppData to start over again (it's taking a while )

[pirloui]

My first post so first off all, thanks for great software with great prospects.

I would appreciate participating with money sometime, in stead of ads / datamining (this project will have to generate some revenue at some point?), but enough of that.

I join "second_coming" in the large folder testing.

I have several folders of 10GB and 100GB synced correctly on a local network. Now adding a 500GB folder. (V1.1.15)

Next step will be moving one of the two machines from Asia to Europe. There it will function as a mirror for the local office. (both sides have automatic backup just in case) Hopefully the shift in time zone will not cause it to resync everything.

Will keep updated in this thread if fine with second_coming.

[GreatMarko]

I would appreciate participating with money sometime, in stead of ads / datamining (this project will have to generate some revenue at some point?)

Please read the Unofficial FAQ entry, "Will BitTorrent Sync remain free, or will they start charging for it once it comes out of alpha?"

Hopefully the shift in time zone will not cause it to resync everything.

BitTorrent Sync references file timestamps back to UTC, so no, in theory, moving a server to a different continent will not trigger a complete re-sync of everything

[pirloui]

Thank you for the reply.

I had seen the entry about BTSync remaining free (as in current version won't expire?) but thought future versions might need to bring income. As I've looked for a long time for these kind of solutions, and this is the closest to what I want, I'd be happy to participate in order for the software to be kept developing. Anyhow, I'm obviously grateful for BT, inc 's good will.

As for UTC, it's a good thing indeed. I'll turn off BTSync autostart to make sure the system is set to the correct timezone first though.

[GreatMarko]

As for UTC, it's a good thing indeed. I'll turn off BTSync autostart to make sure the system is set to the correct timezone first though.

I wouldn't get too hung up over timezones - what's more important is to ensure that each device's time is accurate within in the timezone its running in.

For example, if one device is currently showing 2 minutes past a particular hour, and another device is currently showing 8 minutes past the hour - obviously one of these clocks is either 6 minutes too fast/slow - this could cause Sync an issue when determining which file is to be considered "newer" when syncing!

So my advice; don't worry about the timezone - instead focus on getting an accurate time on each clock!

[pirloui]

That should be OK then, they all get the time off the net Thanks for the help.

This isn't the place for support I guess, but just as a comment since the thread is about this:

The larger folder reports 594GB for 174K files in the finder info, while BTSync indexes it as 553GB for 164K files.

I tried restarting BTSync a couple of times, (it had stopped at 200GB something, continued after restart) but this number is the highest it would get.

I guess there is some FS overhead, and maybe hidden files from Windows metadata (its a OSX 10.6 server) though this sound a bit much of a difference. Anyone an opinion or suggestion for test? I'll still start the sync over the week-end, hoping it finds some more files by then.