How does Bittorrent Sync deal with multiple copies of the same file?


mr.canada

Recommended Posts

Hi - can anyone explain how Sync deals with multiple copies of the same file?  I mean, say I have a folder with 10 subfolders, and for some stupid reason, the same large file exists in all 10 subfolders.  So i have 10 copies of the same file.  Then I tell Sync to sync the parent folder.  Does it transmit the same file 10 times?  Or is it smart enough to send it once, then fill in the blanks at the destination by duplicating that one file 9 more times?

Here's my scenerio - a fileserver with 6TB of historical data.  It's 1000's of large Photoshop images.  Folders are organized by project, year, month.  Even though it totals 6TB, it's really only about 0.5TB of unique data, because most images are duplicated throughout the folder structure.  

As a matter of workflow, staff regularly create more and more copies of these files by placing copies of them into this month's working folder.  So on one hand, every day, lots of new data is created (like, many many gigabytes - more than is reasonable to sync with a modest internet connection).  But on the other hand, very little new data is created each day, because most new files are just copies of preexisting ones.

Is Sync for me?

Link to comment
Share on other sites

All 10 files will be synced as they are. These are different items both for file system and Sync, so Sync takes each of 10 items and syncs.  It's like copying the parent folder from one location to another - all the 10 files will also be copied there, right? Same with Sync - it cannot and doesn't know which of 10 copies you need, which files there are real and which are fake, these are just data bits for Sync. 

Link to comment
Share on other sites

I think the original poster was thinking that because the identical files would (probably) have the same hash, Sync might be smart enough to send the data across the network once, but put the file in all 10 locations.  This is sort of similar to how Sync keeps hashes for files in the archive and will not resend data over the network if a matching file is found in the archive.

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.