The Types of Things People are Using SYNC for

[fongaboo]

(Forgive any goofiness in the wording of this post. I just wrote this up for a class I'm taking and just felt like cutting-and-pasting)
 

I work as a Graphics Designer at a 24-hour TV station. But with my past IT background, I often consult on IT projects that directly concern our department. As we started to plan a full rebranding of our station, including redesign of all graphics and animations, we had to deploy new design workstations and implement new ways of organizing and storing project data.

 

We received four workstations, each with a 6TB storage volume in a striped RAID-0 configuration. This meant that the volume was very fast, but had no fault tolerance. If one drive in the set went down, all the data would be lost. How could we take advantage of the speed of the volume without risking data loss?

 

Another question was how to manage projects made by different designers on different workstations. If one designer had to access a project or graphic file made by another designer, how could we prevent them from having to manually hunt for them on three other workstations' file shares?

 

A central server of some sort - perhaps a standalone NAS - would perhaps fit the bill, except then we'd lose the speed advantage of reading/writing to the internal RAID volume. In this traditional client-server scenario, we'd be limited by the speed of the network connection and that NAS.

 

To get the best of all worlds, I actually turned to a peer-to-peer model, in the form of a free software package called BT Sync. Recently released by the creator of bittorrent, it uses the same infamous protocol to create a set of shared folders that sync with each other across devices in a similar fashion to services like Dropbox.

 

But unlike Dropbox, it uses no central server. The size of the files we'd be manipulating (ie. uncompressed HD animation files, etc.) are way too large to be sending out onto the Internet to some central server and then back again.

 

With BT Sync, we created a shared 'secret' in the form of a hexidecimal number that we install on each machine's BT Sync client. The four machines then collectively form a 'swarm' in which each machine is both a client and a server.

 

When any one designer writes a new file to their drive, the BT sync software is almost instantly aware of it and it works with the other machines to make identical copies on each one. The manner in which the bittorrent protocol accomplishes this also saves on LAN bandwidth. The machine on which the file is created will make a whole copy to a second machine, but from there, those two machines will collaborate together to each copy a portion of the file to a third machine. By the time we reach the fourth machine we have three other machines each copying roughly a 1/3 of the file data concurrently.

We've also solved the problem of the lack of disk redundancy in the individual machines by effectively keeping THREE real-time backups of all data. If one machine's RAID volume ever dies, the data can be easily replicated automatically as soon as the volume is replaced and put back online.

The designers get to interact with what effectively presents itself as a single collective set of files and folders, yet do so with the full speed of their internal RAID volume.

So I'd say that peer-to-peer architecture has been a win-win setup for up our group.

 

[haberjr]

I have set up sync folders on a Mac mini at our home and while traveling had our laptops sync with the Mac mini as backup (read only) using bit torrent sync. Worked flawlessly.

 

I am considering the following setup. Have our home computers (3) sync their critical files on the Mac mini that serves as  a backup. Install Crashplan on the Mac mini and pay for a 1 computer backup on their Crashplan central. In this way I don't have to pay for a family plan, while at the same time all the computers, including the laptops will always be offsite backed up. 

 

So step 1 is to use bit torrent sync critical folders to my central Mac mini for desktops and laptops

Step 2 is to use crash plan for that computer to backup to Crashplan central as insurance for theft, fire paying only for 1 computer

[kevinsky25]

I'm using Sync to synchronize my 5 GB school projects and other documents. I'm also using it to sync a 20GB truecrypt container with personal data such as photo's and videos but also tax documents and so on. Syncing across Windows, Mac and Linux

[DesignMongrel]

I'm using it to have my own secure server (obviously) but to ween myself off of DB/Drive and only use those for friends and family who need files. I was looking for a solution and when I read about the Raspberry Pi and BTSync together, I made it happen. A few sleepless nights and only missed one class due to being up till 5am I finally got it working.

 

I am a graphic designer, school newspaper photographer and student and having my own cloud lets me walk around with less weight on me, as I carried a 1TB hdd on me, albeit small but still less of a risk of damaging it by having it at home and always connected.

 

I'm waiting on a few issues to be resolved, such as my Android phone and being able to sync on an external SD card and also being able to "see" the files without having to export them unless needed to, on my phone since it has much more limited space (unless I'm misunderstanding the read-only option or there's another solution I haven't read about?). My work around has been to make an empty folder on my cloud and just transfer over files manually when I'm home.

Other than that, BTSync has been a lifesaver. Hopefully it just gets better from here on out.

 

Devices connected through my cloud

Windows 7 PC (Home)

Macbook Pro Retina (School/Work)

Samsung Galaxy S4 (Everywhere I'm at)

HP Touchpad with Cyanogenmod (Anywhere)

 

BTSync on a Raspberry Pi Model B, running Linux, low power always on cloud solution.

[jona303]

Hi, I'm using it to sync my local document root for apache between my laptop and my desktop. For now i'm really confortable with this solution

[Nefarious]

Utilizing low cost Raspberry Pi's and BTSync, I have been able to create a small private network with systems located across the country. I can keep my files backed up and have no worries about my stuff being accessed from the cloud without my permission.

[fscheps]

I am syncing a complete offsite backup from my Synology DS1513+ to an external hard drive I sent to my dad and he connected to his computer (11.000 KM away .

 

This way I get an offsite backup and he gets to have the files I want to backup if he wants to use them too.

 

It would be great to have an official Bittorrent Sync client for Synology NAS and to be able to sync subfolders of a shared folder. Then the app would 100% perfect.

[DanPrs]

I use this to sync/send music to my phone from my PC. I also use it to back up my Pictures and Music folders from my PC to my FreeNAS server. 

I also gave my sister access so that she can sync/back up her data to my server. She's in FL, I'm in NY. Works great. 

 

I wish there was a back up and recovery function for mobile (Android) devices. 

[MisterRider]

Is anyone using BTsync as a backup solution? For example, by modifying the sync_trash_ttl variable? I'm wondering why I would run an extra backup program (ex: BackupPC, Syncrify, etc) for my client machines , when most of my folders are duplicated on my desktop and two laptops... 

Thoughts?

[Doc Green]

Is anyone using BTsync as a backup solution? For example, by modifying the sync_trash_ttl variable? I'm wondering why I would run an extra backup program (ex: BackupPC, Syncrify, etc) for my client machines , when most of my folders are duplicated on my desktop and two laptops... 

Thoughts?

I am actually using it in the that way for some sets of my private files, and it is pretty handy for small office files and so on. Still, I can think of scenarios where this will not suffice as a backup, although to my mind these scenarios are a negligible risk in a private environment. Physical harm can be prevented by offsite Sync instances. Trojans or viruses attacking my datas integrity on all devices seems to be a low risk either at the moment. On the other hand Sync is still beta so I would not fully trust it. Think of the reappearing deleted files, files not fully synced or conflicting. It happens very rarely, but you would not want to verify the datas integrity all the time. And for frequently modified folders, it is hard to find the version you want to revert to. In addtion I am having an always on device, it keeps all the versions of the files in Synctrash, but not everybody wants that or can afford that.

For now I use it as an addtional backup service, it serves its purpose as a first backup instance and proved to be very useful.I have already encountered a hdd crash on a laptop since I am using Sync. I just had to install a new hdd, reinstall the OS (in this case using an image of a fresh installation), enter the secrets of my shares and I was good to go again in less then an hour. I did not have to deal with older file versions or old backups. No other solution I know of would have been that easy and painless, Dropbox and the others out there would charge a lot of money for the amount of data and it would have been a LOT slower.

However I still use offline backups on external storage devices, better save then sorry

So long

Doc

[hungarianhc]

Like others, I have a NAS. I have BTSync installed on my NAS (custom FreeNAS build) and I have folders sync'ing on my computers and on my wife's computers. Most of the stuff I have sync'ing are just my own files, but my wife and I have a shared folder that is sync'd up that we both work on, and it works well for that, other than the fact that the FreeBSD version seems to be crash-prone at the moment.

[PacketMan]

I am using Sync to replicate home security camera system pictures to my various devices including hand-held devices. The system dumps pictures to my NAS which is SMB based. In my PC I that that SMB share and point Sync at it. When the system dumps some pictures to the share, Sync will replicate them to my hand held devices.

 

In the event my alarm company calls, the intent (hope) is that the pictures of the bad guys are already on the device. If this works well, then I just might turn off email based notification.

 

This is not working as well as I hope, only because the security camera system is a cheapo, and it hard to tune the alarming thresholds. I turned the sync folder scan from 600 to 60 seconds, but today for example, my family came home, causing the system to spit out over 2000 pictures in a couple hours. Sync on my PC didn't crash, but it sure had the CPU pinned until I cleaned out the folder. 

 

I think if I can buy a top quality security DVR sytem, and 'tune' it the way I want, Sync will be real sweet here.

 

 

Also, I have anothe Sync folder on my handhelds, my wifes, and kids. We each snap up sponteansous moments and poof they appear on everyone elses devices, plus back at home base PC for easy tranferring to a main NAS, and also to backup. Sweet!

[palampam]

-video games saves/musix between my laptop and my desktop

[stedlj]

No more drop box!

I have my own cloud using 2 Raspberry Pi's, one at home the other with family out of town. I added Hamachi VPN to them so I could update/manage both of them with out going to their house. Showed a friend how to do it and now he has a simular setup.

[ManipulatingBastard]

Here's my story, and why I decided to spend time writing such an elaborate post.

 

I work as a creative Business Consultant.

 

And by "creative", I mean a Business Consultant whose work differs alot from that of most Business Consultants - many of whom you've likely already met (in particular if you work in a large organization) during one or more of your projects.

 

Allow me to explain:

 

I am indeed what you would call a "regular Business Consultant", and I travel alot all over the world. More specifically, I work professionally within the discipline of Enterprise Architecture whereas my personal specialty lies within the area of Process Architecture.

 

The reason I use the term "creative", is because I am also a graphics designer. This is a competency I have acquired long before I got to become a Business Consultant.

 

And being a good graphics designer as well as being proficient in Enterprise Architecture (and all the other disciplines it involves) allows me to create very persuasive graphics of "how things work", "when are they used, for what purpose", "who uses them", "in what context are they used", etc.

 

In essence, you could say that, instead of killing CEO's with ridiculously cumbersome and boring elevator pitches... I draw them pictures!

 

And that's where Bittorrent Sync enters the fray!

 

Because damn! I have just fallen in love. It is one of the few software products that I would actually buy, should the developers decide to charge payment.

 

Here's the technicalities:

 

Like I said earlier, I travel alot. And sometimes I don't.

 

An example; 2 weeks ago, I just got home from another country, and when I get home, alot of my files in "specific folders" on my workstation were outdated. Because I;

 

1) created new files when I was in another country.

2) I have updated and modified existing files while in another country.

3) I have deleted files that I deemed was no longer necessary to keep and/or protect.

 

And being able to synchronize so easily, the contents of a folder from one device to another, is really fucking epic to people like me!

 

But it's not just the ability to synchronize! It's about to synchronize files across devices WITHOUT having to worry about the safety of your data!

 

"Heartbleed" is a great example. First thing I did when one of my really tech-savvy friends told me about it was to launch unto Dropbox and change my password. That's the real problem with the cloud. Your data security is only as strong as the infrastructure protecting it, ie. "heartbleed" was a data breach. You cannot breach the security of Bittorrent Sync unless you do it live and in real-time.

 

PS: Pardon my english. We danish people are very liberal, I apologize if it offends anyone!

 

PPS: Sorry for this wall of text. But perhaps someone can take away some nuggest of knowledge and/ord wisdom from this. Take care all!

Edited April 12, 2014 by ManipulatingBastard

[exhibit]

I just use it for sync music files for my DJ laptops... The always-on machine is a Raspberry Pi hooked up with a 2TB drive.

[NobleDeadOrg]

Authors Barb and J.C. Hendee now use BTS throughout the household network and while working elsewhere to produce multiple isolated versions of all past works, works in progress, and needed support files. This has become the first tier in their three tier backup for critical data... and all use of cloud storage (risky at best for intellectual properties) has been terminated.

Thank you BTS!!!

Edited April 18, 2014 by NobleDeadOrg

[zkyevolved]

Hi guys! Let me ask you a question. Is the Raspberry Pi a good device to keep as a "cloud" storage for BT Sync? I have an Android tablet and I set it up to be read only but it gave me a lot of issues. I tried it on my Galaxy Note 2 and it also went out of sync frequently and wouldn't update. Read Only and Read Write options made no difference. Wifi was always on, etc. 

 

Basically, what I'd like to go is get a device that would sync my documents as a backup and distribution to other devices. For example, I would edit a document, save it, and turn off my computer. Go to work, and turn on my computer there or my laptop and would like to continue editing. I saw this is not possible with Android with Autosync on... It just didn't work very well. So Android is out of the picture. How about a Raspberry Pi model B with either an SD card or USB drive? I'm looking for something a little bit more reliable than just Android. I'm looking for something with very low power consumption, as well. Any thoughts? 

 

Thanks in advance. Kind regards!

[RomanZ]

@zkyevolved,

 

There are a lot of users on forum reported to run BTSync on RPI. Couple of peculiarities you should be aware regarding BTSync on RPI:

 

1) CPU power. BTSync uses CPU extensively for 2 main things: hash data (when new files are added, new files are coming from network, etc), and data encryption / decryption (especially if peer is an encrypted node). As RPI is a cheap and low-power platform, you should not expect an overwhelming performance,

2) Power issues. Note, that RPI is powered via USB and consumes no more than 500mA (as according to USB 2.0 specs). The external drive you connect is strongly advised to have it's own power supply or you can get a whole bunch of weird and strange issues with it.

3) Permissions. Be sure that you set up your permissions on Linux clearly and simple or you might get a lot of issues.

 

That's something I'm aware of.

 

Maybe other forum users can share their experience on using RPI with BTSync.

[Moe]

The Rpi works exactly how I wanted to work. I run Transmission and BTSync on it and I have an external 1.5TB HDD connected. Yeah sure the cpu power isn't that high and it takes long to hash files for the first time but for files up to 10 MB it works like a charm (Photos, large word documents, PDFs)

[bwainscott]

I also have a RPI as my "always on" sync system, with an external 1TB drive (attached via a powered hub -- the RPI definitely does not supply enough power to connect it directly).  It works quite well, but I must confess I do not tax my system at all.  I sync our digital photo archive, recipes, my music, the pictures from both my phone and my wife's, and not much else -- and none of this changes often.  I have 2 laptops (one Windows 7, one Linux), two Android phones, and the RPI.  The RPI is essential since the others can never be guaranteed to be on at the same time.

 

I got tired of logging in to the RPI to update the btsync client with each new release, so set up a cron job on a 10 minute schedule, that checks a particular share for a new "btsync_arm" executable.  If it finds it, it stops btsync, installs it, deletes it from the share, and restarts btsync.  I only had an opportunity to try it for the first time when going from 1.3.87 to 1.3.93, but it worked great -- I just downloaded the btsync_arm to the share on my laptop, and 15 minutes later my RPI was running the new version (and had deleted the file from the share).  Computers are so much fun when they work correctly!

[DesignMongrel]

I also have a RPI as my "always on" sync system, with an external 1TB drive (attached via a powered hub -- the RPI definitely does not supply enough power to connect it directly).  It works quite well, but I must confess I do not tax my system at all.  I sync our digital photo archive, recipes, my music, the pictures from both my phone and my wife's, and not much else -- and none of this changes often.  I have 2 laptops (one Windows 7, one Linux), two Android phones, and the RPI.  The RPI is essential since the others can never be guaranteed to be on at the same time.

 

I got tired of logging in to the RPI to update the btsync client with each new release, so set up a cron job on a 10 minute schedule, that checks a particular share for a new "btsync_arm" executable.  If it finds it, it stops btsync, installs it, deletes it from the share, and restarts btsync.  I only had an opportunity to try it for the first time when going from 1.3.87 to 1.3.93, but it worked great -- I just downloaded the btsync_arm to the share on my laptop, and 15 minutes later my RPI was running the new version (and had deleted the file from the share).  Computers are so much fun when they work correctly!

 

Could you elaborate on how to do this? This would be very useful.

@zkyevolved,

 

There are a lot of users on forum reported to run BTSync on RPI. Couple of peculiarities you should be aware regarding BTSync on RPI:

 

1) CPU power. BTSync uses CPU extensively for 2 main things: hash data (when new files are added, new files are coming from network, etc), and data encryption / decryption (especially if peer is an encrypted node). As RPI is a cheap and low-power platform, you should not expect an overwhelming performance,

2) Power issues. Note, that RPI is powered via USB and consumes no more than 500mA (as according to USB 2.0 specs). The external drive you connect is strongly advised to have it's own power supply or you can get a whole bunch of weird and strange issues with it.

3) Permissions. Be sure that you set up your permissions on Linux clearly and simple or you might get a lot of issues.

 

That's something I'm aware of.

 

Maybe other forum users can share their experience on using RPI with BTSync.

 

These were the least of my worries, lol. The main issues I ran into were figuring out how the damn permissions worked, and then setting up a work around since every device connected would in turn download whatever you have in your directories.

 

So I just set up folders within the main directory for my Macbook and my mobile devices (android phone & tablet, and even iPad 2 to test out BTSync on iOs). Now I won't find my devices trying to download all my huge files unless intended to.

 

On a side note, will they ever implement an image only viewing setting for mobile devices? I.e. Dropbox and the way you can see files in a folder without having them download to your device unless intended to, I guess viewing it as an "image".

[bwainscott]

Could you elaborate on how to do this? This would be very useful.

 

I'll post below the script that I run via cron.  It is of course specific to my setup, which looks like this:

 

On my RPI, the directory /home/pi/BTInstall is the share where it expects new executables to show up.  It expects them to be named "btsync_arm-X.Y.Z.tar.gz".  My running executable is in /usr/local/bin/btsync, and is a symbolic link to the actual executable, which is /usr/local/bin/btsync_X.Y.Z

 

I use the same script on my laptop, which has similar setup but different "install from" directory.  The script takes a single argument: if it == "pi" then it does everything silently and with the appropriate settings for my RPI.  Otherwise it is running on my laptop interactively, and it lets me know what it is doing.  Here is the script:

 

#!/usr/bin/python

# Takes one optional argument.  If "pi" then am on my pi, else on laptop

import sys

import os

import stat

import time

import signal

#   

if(len(sys.argv) < 2 or sys.argv[1] != 'pi'):

  onpi=0

else:

  onpi=1

#     

# Directory to look in, and filename to look for

#     

if(onpi):

  SYNCDIR="/home/pi/BTinstall"

  LOOKFOR="btsync_arm"

else:

  SYNCDIR="/home/brian/Downloads"

  LOOKFOR="btsync_x64"

#

# for chmod call below, to set rwxr-xr-x

#

mode=stat.S_IRUSR | stat.S_IWUSR | stat.S_IXUSR

mode= mode | stat.S_IRGRP | stat.S_IXGRP

mode= mode | stat.S_IROTH | stat.S_IXOTH

#

# Go to where the file should be, and get a directory listing

# Sort it in case there is more than one btsync_* -- install

# the oldest.

#

os.chdir(SYNCDIR)

files=os.listdir(".")

files.sort()

for t in files:

  if(len(t) > 10 and t[:10] == LOOKFOR):

#

    if(not onpi):

      print "Installing %s" % t

#

# expect name like btsync_arm-X.Y.Z.tar.gz (arm or x64)

# this pulls out the X.Y.Z

#

    name=t.split("-")[1][:-7]

#

# See if btsync is running.  If so, stop it, wait to be sure, and set

# flag to restart it

#

    restart=0

    f=os.popen("ps -e","r")

    for p in f.readlines():

      parts=p.strip().split()

      if(parts[-1] == 'btsync'):

        os.kill(int(parts[0]),signal.SIGTERM)

        time.sleep(30)

        restart=1

        break

#

# Put new executable in place

#

    os.remove("/usr/local/bin/btsync")

    os.system("tar -xzf %s -O > /usr/local/bin/btsync_%s" % (t,name))

    os.chmod("/usr/local/bin/btsync_%s" % name,mode)

    os.symlink("btsync_%s" % name,"/usr/local/bin/btsync")

#

# Remove the file we installed, and optionally restart btsync

    if(restart):

      os.system("rm %s ; /usr/local/bin/start_btsync" % t)

    else:

      os.system("rm %s" % t)

    sys.exit(0)

 

[Sue D. Nymme]

I'm using Sync for several things, but there's one small, simple, fantastically convenient use that I'd like to share with the community.

I use a password manager (Password Safe, but this concept would work with any of them) to store all my myriad passwords. I use BTSync to replicate the database on my home desktop, my laptop, my Android phone, my partner's phone, and my work desktop. Not only are all our passwords securely available on all those devices now, but if one of us updates a password or creates a new login to some new online service, it's instantly available to us both wherever we are. For example, I created a new login at me health insurance company's website today. That's something that we both need to be able to access any time.

Highly recommended.

[vlmath]

Of course sync between work (workstation and personal server) and home.

I'am planning using it for job distribution between home make cluster (data generation, and instant backup). more than 5 To planned ^^