Delta Updates

[Instine]

If VM disks could be synced by diff (not the whole file), this would be a new and very powerful capability, which is hard to find elsewhere or create securely between networks. I've heard a few saying this on here. I've heard staff say this wasn't important. I'm sorry to be blunt, however, it's not about what features are important to Sync staff, it's about what's important to users. I'm a user. It's important.

As a software architect I gotta say, both for dev machines that stay synced across my multiple work environments, and for service machines which could far more quickly update not just source but the entire host OS and its install software, it would be hugely useful and very valuable.

Please don't dismiss this again and point people to the FAQ. 

You may think there's not many software architects out there, but we make the services that  millions of users use every day. You make our lives easier, you make all those users happier. I'm not saying I'm a big deal (I'm not I promise), but this could be a big deal in terms of 'disruptive' progress in big datacenter and inter-datacenter management if it had delta capability.

Otherwise, thanks for a wonderful FOSS tool that is feeling more and more solid and useable every time I update. Great work guys!

[GreatMarko]

Please don't dismiss this again and point people to the FAQ.

...and I would also point you in the direction of the FAQ, specifically the answer to the question: "When a file changes, does BitTorrent Sync transfer the entire file again, or just the part that's changed?":

 

"Files smaller than 4MB are transferred again in their entirety when changes are detected.

Larger files, however, are split into 4MB "chunks" of data, if one chunk changes, but the others don't, only the chunk that's changed will be transferred (instead of the whole file again) ...however, If you add one byte in front of the file, all chunks will be shifted and Sync will not be able to detect shifted data."

 

So in terms of large files, Sync doesn't automatically re-transmit the entire file if it can possible help it, and will only transmit those chunks of data that have changed (assuming other chunks haven't changed as well).

 

Also, you may be interested in this previous discussion on syncing changes/diffs.

[Instine]

Great news. Thanks. It does rather well illustrate why a generic'go read the FAQ'answer sucks though. FAQ can get out of date for one. So not trusted. Also diffing blocks isn't the same as diffing bytes. By a long shot. Esp regarding VMs. So it shuts down discussion before it's done. Sorry to be so critical when you are doing great work.

[GreatMarko]

FAQ can get out of date for one. So not trusted.

Well, it's a shame you "don't trust" the FAQ's, because if you take a moment to look at the "Last Edited" date at the bottom of them, you'll see that they ARE regularly maintained!

 

Also diffing blocks isn't the same as diffing bytes. By a long shot. Esp regarding VMs.

 

I would think comparisons are done on a "block" level rather than a "byte" level in order to improve speed/performance/efficiency and reduce CPU load.

[ChrisH]

If you are trying to sync live VMs (both source and target) you are bound to fail anyways. No running VM will like it very much if some external program arbitrarily changes blocks (or bytes) on the storage layer.

 

If the target VM is NOT running it should not matter whether you compare bytes or blocks. You still can't rely on the target being consistent as long as data on the source is changed and synced.

 

Hyper-V for instance has an extra copy-on-write mechanism for VM replication to make sure that all changes are synced in the correct order. And even there the target VM has to be shut down while replicating.