Entire file or parts of file?


Castor

Recommended Posts

Does SyncApp upload the entire file or only parts of file that are modified? I have a 11 MB zip file that got uploaded entirely when I made a change in one of the spreadsheet file in the zip.

BTW, the zip file i used is password protected and has the extension .zipx. It has only .xlsx file in it which was also password protected. The xlsx file has many sheets. I made the change in one of the sheet.

Link to post
Share on other sites

Dropbox does that, for couple of reasons: it saves them money on harddrive space and network traffic. Which actually makes perfect sense. In case of Sync, it might be faster to just transfer the file, rather than calculate difference and then apply it to the file.

Having in mind that all files are binary nowadays, this is not as effective as difference for big text files.

Anyway we have this in mind, however importance of this feature is questionable, of course if leave aside marketing noise about this feature.

Link to post
Share on other sites

You really think, this is a questionable feature? oO

Delta copying (or syncing) saves bandwith & time and both things are very valuable things (at least in most countries where your upstream is rather limited). I've used rsync for my cloud backups so far (under Windows *urgs*) just because it's so efficient because of it's delta copying feature...

Link to post
Share on other sites

Don't get me wrong, we do understand importance of the diff and its advantages. And actually Sync support delta copying if file size stays the same.

However implementing rsync like diff in distributed environment without dedicated server, is kind of complex task.

Link to post
Share on other sites
  • 2 months later...

If you read carefully, Sync already has diff that works on block level and fits perfectly with TrueCrypt. Since it is drive mirror of the same size, it is changed by blocks. Sync will not transfer blocks that are similar between peers.

The rsync diff is used when you add/remove some data from a file. Sync doesn't handle this.

Link to post
Share on other sites

The rsync diff is used when you add/remove some data from a file. Sync doesn't handle this.

Ah, I didn't completely understand until you said that. Now it makes sense. I was trying to sync back an entire VM and it ended up transferring the entire 19GB, which filled the disk on the other end and broke the transfer -- luckily leaving the old intact file in the trash folder. Now I'm wondering if I could get the diff functionality you mentioned simply by setting the VM to allocate the entire virtual disk. I will check back if I find success...

Link to post
Share on other sites
Now I'm wondering if I could get the diff functionality you mentioned simply by setting the VM to allocate the entire virtual disk. I will check back if I find success...

I'd be interested in your results - I'm hoping to use Sync to synchronise all my virtual machines between two computers where each VM is around 25Gb.

Link to post
Share on other sites

Well, it took a few days to get going. I originally transferred the vmdk to my home machine using BTSync prior to my post. I tried syncing with the dynamic disks, but it wanted to transfer the whole 20GB files. I don't even have 20GB of free space on either host disk, so that failed. That's when I saw kos13's post about the file sizes.

So, to get matching file sizes, I converted both vmdks to fixed disks with the VboxManage tool. Their sizes and file names were identical. The vmdk on my home machine had an earlier modified date than the vmdk on my work machine. Upon trying the sync, BTSync still wanted to transfer the entire files. Additionally, it wanted to send the older vmdk up to my work machine to replace the newer one. Needless to say, the whole process failed due to lack of free space. I managed to recover the vmdk too.

My next step is to ditch the home fixed vmdk and resync the work one down, then try to make changes to the home one and see if it transfers only chunks. It's probably going to be another day or so before that experiment is complete.

Link to post
Share on other sites

Aaand... that didn't work either. This time I used BTSync to transfer down the fixed vmdk from my work to home. After using the work VM and shutting down, BTSync proceeded to sync down the entire 20GB file. So it looks like the current version of BTSync does not support this use case. I'm confident that it will be able to do so eventually.

Link to post
Share on other sites

Delta sync is one of two features that I'd love to see added to BTSync. +1 for me. The other feature is one of a sort of global hash table where two hashes of two files that are identical wouldn't have to be transferred because that block of data is already at the destination. According to this thread, they're going to do it:

Perhaps these two features are related. I'm not a developper so I don't know.

What's the roadmap for proposed features? When can we expect a version 1.0 release?

Link to post
Share on other sites
  • 2 months later...

Archived

This topic is now archived and is closed to further replies.