Scorp1us Posted April 23, 2013 Report Share Posted April 23, 2013 So RSync is good at transferring files, and the differences between them. Meaning added, deleted or changed blocks. And given that you create/modify at one location at a time. Rsync seems a better fit.Bittorent is engineered for getting 1 file from many sources. You're not going to have many sources in your private cloud. So why should we use bit torrent. I can't think of any reasons to, but many not to:- ISP throttling/blocking.- RSync is more efficient. Quote Link to comment Share on other sites More sharing options...
Harold Feit Posted April 23, 2013 Report Share Posted April 23, 2013 If RSync is better geared for your use case, use it.BitTorrent Sync is intended as a replacement/supplement to Dropbox and other cloud storage systems.Additionally, BitTorrent Sync is many to many file transfers, where rsync is one to one. When dealing with large numbers of systems to keep synchronized, BitTorrent Sync scales better once the origin system's upstream is saturated. Quote Link to comment Share on other sites More sharing options...
GreatMarko Posted April 23, 2013 Report Share Posted April 23, 2013 You're not going to have many sources in your private cloud.I beg to differ! I currently sync between 7 devices (including two always-on WHS's)So why should we use bit torrent. I can't think of any reasons to, but many not to:- ISP throttling/blocking.1) You can change the ports that BitTorrent Sync uses, as well as choose TCP over UPD - it would be difficult for an ISP to completely block BitTorrent Sync2) You can separately limit the upload/download rates of BitTorrent Sync - so you can easily tailor BitTorrent Sync to your connection speed/available bandwidthSo why should we use bit torrent. I can't think of any reasons to, but many not to:- RSync is more efficient.I may be wrong, but doesn't RSync only transfer a single file at a time? BitTorrent Sync can transfer multiple files concurrently!Give BitTorrent Sync a go! You'll then be able to more accurately determine its capabilities/performance when compared with RSync! Quote Link to comment Share on other sites More sharing options...
Scorp1us Posted April 23, 2013 Author Report Share Posted April 23, 2013 True, but it would be interesting to see in some tests who performs better. Given that most changes are small incremental ones, rsycn should be competitive for a large number of test cases.I thing you'd need an organization to generate enough load that rsync starts to falter. Which is fine, but when I hear "dropbox" I think individual. Quote Link to comment Share on other sites More sharing options...
Scorp1us Posted April 23, 2013 Author Report Share Posted April 23, 2013 But RSync is only available for Unix based systems!False. There are no algorithms that can't be done on Windows if they are done on Unix https://www.itefix.no/i2/cwrsync (Python: http://blog.liw.fi/posts/rsync-in-python/ )it would me interesting to see if you could replace your torrent based solution with rsync and see which one comes out ahead. Quote Link to comment Share on other sites More sharing options...
kos13 Posted April 23, 2013 Report Share Posted April 23, 2013 Dropbox is rsync made for people. It is nice and perfect. Sync is about moving huge data, try to move 300Mb file via Dropbox and Sync and you fill the difference.Sync never planned to be a replacement for Dropbox/rsync, while it has some features that looks the same. If you need to send parents 500Mb of your kid's photo/video - you don't need a rsync, right?We already have some diff based transfer, however not rsync like. Sync can't detect shifted blocks of data. If block wasn't shifted, it will not be send again. Quote Link to comment Share on other sites More sharing options...
kos13 Posted April 23, 2013 Report Share Posted April 23, 2013 Forgot your original question I am not aware of any rsync based solution that is serverless and distributed. Sync is truly server-less and distributed, this is why it is not rsync based. Quote Link to comment Share on other sites More sharing options...
Scorp1us Posted April 23, 2013 Author Report Share Posted April 23, 2013 As an aside, I just find it amusing that it is 2013, 50 years after the first filesystems and 20 years since the took off web we're still inventing ways to move files between computers. Quote Link to comment Share on other sites More sharing options...
kos13 Posted April 23, 2013 Report Share Posted April 23, 2013 Yes, this was a surprise to us too, that Sync is the only effective way to send 1Gb between 3 computers. Quote Link to comment Share on other sites More sharing options...
Scorp1us Posted April 23, 2013 Author Report Share Posted April 23, 2013 Forgot your original question I am not aware of any rsync based solution that is serverless and distributed. Sync is truly server-less and distributed, this is why it is not rsync based.It depends on how you define "server" RSync can operate on it's own by comparing two files. If one of the files is remote, it is heavily benefited by being able to calculate the checksums locally, then comparing them. So yes, there may be a "server" on the remote system that reads the file and transfers the checksums. Then once that is done, the server accepts and applies the incoming deltas. Quote Link to comment Share on other sites More sharing options...
Scorp1us Posted April 23, 2013 Author Report Share Posted April 23, 2013 Yes, this was a surprise to us too, that Sync is the only effective way to send 1Gb between 3 computers.This is an interesting claim. No matter what protocol, you have to send 2gb to have the same file on all computers.But I am interested in what happens next. If you modify the file, what has to be transmitted? Perhaps it is just publish/download. But once distro I used only provided Rsync of ISOs so you'd only get the 20mb or so of changes. How does bittorrent respond?To me one of the advantages of Bittorrent is when you have lots of seeders. But if you have a microcloud of 3, I don't see BT doing nearly as well. Quote Link to comment Share on other sites More sharing options...
kos13 Posted April 23, 2013 Report Share Posted April 23, 2013 This is an interesting claim. No matter what protocol, you have to send 2gb to have the same file on all computers.But I am interested in what happens next. If you modify the file, what has to be transmitted? Perhaps it is just publish/download. But once distro I used only provided Rsync of ISOs so you'd only get the 20mb or so of changes. How does bittorrent respond?We made a first step in this direction. We understand importance of rsync, but it will be next step. To me one of the advantages of Bittorrent is when you have lots of seeders. But if you have a microcloud of 3, I don't see BT doing nearly as well.You are right, this is one of the advantages. Another one is uTP and direct connection between peers. To give you real example, try to pass 500 Mb to your friend. If you don't mind, compare the speed of Dropbox and Sync. This will surprise you. Quote Link to comment Share on other sites More sharing options...
m1bxd Posted April 23, 2013 Report Share Posted April 23, 2013 Get around the problem of firewallingCross platformI bet Android and iPhone get their own apps over timeIt'll be so simple sh33pl3 can use itSo RSync is good at transferring files, and the differences between them. Meaning added, deleted or changed blocks. And given that you create/modify at one location at a time. Rsync seems a better fit.Bittorent is engineered for getting 1 file from many sources. You're not going to have many sources in your private cloud.So why should we use bit torrent. I can't think of any reasons to, but many not to:- ISP throttling/blocking.- RSync is more efficient. Quote Link to comment Share on other sites More sharing options...
kewi Posted April 23, 2013 Report Share Posted April 23, 2013 Why bicker about it? rsync is still there as it always has been. Use the appropriate tool for the task at hand. Quote Link to comment Share on other sites More sharing options...
Scorp1us Posted April 23, 2013 Author Report Share Posted April 23, 2013 Why bicker about it? rsync is still there as it always has been. Use the appropriate tool for the task at hand.Very much so. But I suggest you take it further. Use the appropriate protocol for the task at hand. If BT would be inferior, switch to Rsync. Then you have one tool that can work better in more situations. But the important distinction is that the computer knows, so the computer picks. The choice can be hidden from the user and they just see lots of win. Quote Link to comment Share on other sites More sharing options...
zyndarius Posted April 23, 2013 Report Share Posted April 23, 2013 Good questions the one Scorpius asks. As far as I know, if BTSync uses the bittorrent protocol, an approach to detect file variations would be something like a systems that "recalculates" the chunks in which a file is partitioned and then compare the chunks with variatons? Is that feasible? Quote Link to comment Share on other sites More sharing options...
kos13 Posted April 23, 2013 Report Share Posted April 23, 2013 Good questions the one Scorpius asks. As far as I know, if BTSync uses the bittorrent protocol, an approach to detect file variations would be something like a systems that "recalculates" the chunks in which a file is partitioned and then compare the chunks with variatons? Is that feasible?BTSync does that. However this is not how rsync works. If you will add just one byte, all chunks will be changed. You need to detect shifted chunks. Quote Link to comment Share on other sites More sharing options...
Scorp1us Posted April 23, 2013 Author Report Share Posted April 23, 2013 Well then that's cool! Quote Link to comment Share on other sites More sharing options...
Urist Posted April 23, 2013 Report Share Posted April 23, 2013 So, correct me if I'm wrong, but I just spent more time trying to figure out how to use rsync with two dynamic IP addresses over the internet than the time it took me to install Sync on two machines and start syncing. Keeping in mind that these two machines are windows and linux(ARM at that).Rsync is not for everyone. I know my way around a computer and it wasn't just "simple" to do what Sync does out of the box.Sync:me: Download this, install it, use this key.friend: OKRsync:me: can I come over and set this remote backup thing up?friend: This is more trouble than it's worth.Or something along those lines. Quote Link to comment Share on other sites More sharing options...
zyndarius Posted April 23, 2013 Report Share Posted April 23, 2013 So, correct me if I'm wrong, but I just spent more time trying to figure out how to use rsync with two dynamic IP addresses over the internet than the time it took me to install Sync on two machines and start syncing. Keeping in mind that these two machines are windows and linux(ARM at that).Rsync is not for everyone. I know my way around a computer and it wasn't just "simple" to do what Sync does out of the box.Sync:me: Download this, install it, use this key.friend: OKRsync:me: can I come over and set this remote backup thing up?friend: This is more trouble than it's worth.Or something along those lines.Yes, I have to share that opinion too, BTSync offers a near-zero learning curve, at least for the basics. RSync is a very potent application no doubt about that, but it is not for my granma for instance, not even my little brother. So the user adaption on the utilty is a very important matter.With respect to the mechanism that BTSync uses to detect and propagate file changes. The time taken to re-calculate the chunks of the changed file is dependent on the size of it right?. How much would that be for 1GByte, lets say? Quote Link to comment Share on other sites More sharing options...
Scorp1us Posted April 24, 2013 Author Report Share Posted April 24, 2013 That's not a fair comparison. If someone wrote Sync with all the features it has, which uses BT, then a fair comparison is if someone wrote a comparable Sync for RSync. BT and RSync are the protocols, Sync is a app using the protocols. Quote Link to comment Share on other sites More sharing options...
Scorp1us Posted April 24, 2013 Author Report Share Posted April 24, 2013 The problem I have with labeling this thing as a good thing for the "cloud" is cloud files are not the same as general torrents. Torrented files are copies of an original that never changes. With cloud files,, there is a higher expectation that this is a working file. Quote Link to comment Share on other sites More sharing options...
Scorp1us Posted April 24, 2013 Author Report Share Posted April 24, 2013 BTSync does that. However this is not how rsync works. If you will add just one byte, all chunks will be changed. You need to detect shifted chunks.Can you elaborate on what Sync is capable of detecting and how it responds? Quote Link to comment Share on other sites More sharing options...
kos13 Posted April 24, 2013 Report Share Posted April 24, 2013 Sync splits file to 4Mb chunks, if one chunk was changed - it will be transferred. It works well on files that doesn't change the size, for example TrueCrypt volume.If you add one byte in front of the file, all chunks will be shifted and Sync will not be able to detect shifted data. Quote Link to comment Share on other sites More sharing options...
Urist Posted April 24, 2013 Report Share Posted April 24, 2013 That's not a fair comparison. If someone wrote Sync with all the features it has, which uses BT, then a fair comparison is if someone wrote a comparable Sync for RSync. BT and RSync are the protocols, Sync is a app using the protocols.So, IF someone wrote a program that uses Rsync and was as easy to set up and use as Sync is now, the only benefit I see is when syncing two machines. When you add more machines to the mix, the distributed nature of BT is a clear winner. And this is still a big IF someone made a very simple to use Rsync program, which there might very well be.The way I see it, linux is by far a more powerful OS than windows. Why do people use windows(when they have a choice)? Ease of use. People love stuff that just works. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.