[Raspberry Pi] Automated Installation/update And Management Script

[Moe]

A lot of people on this forum have a Raspberry Pi using BitTorrent Sync.

Over the last couple weeks, since the original blog post of Sync + WD + Raspberry Pi, I have been working on a script for everybody to use to install, update, backup and remove BitTorrent Sync on their Raspberry Pi easily without worrying how to actually do that.

And now I am ready to let you guys know about it.

It does exactly what you would think it does: Install, update, backup and remove an installation of BitTorrent Sync using that script. Not more, not less.

 

You can check it out on my GitHub page here: Raspberry Pi + Sync Script

All the instructions and what to do is described there.

 

Hope this will help people that are just starting to use BitTorrent Sync on their Raspberry Pi's.

[dustmite]

Thanks, I'll check it out, its exactly the type of thing I need

[19feet]

Thanks. Great work! Does it also work with Ubuntu 15.10 64bit because I have several PCs with the old version of Tuxpoldo which is stuck on Btsync 1.4.111?

[Moe]

19feet: Sorry, no it does not work on Ubuntu or any other platform other than ARM.

Also the script was made and designed to run on the Raspberry Pi only.

[jake.sadie]

installed but init.d script doesn't seem to be working for me?

 

things I changed: I had to change my user name in the script to match my current user name. i changed the config.json to simply change the port number, so there should be no issues there.

 

installed without errors after that point. now running the start command produces the echo output properly, but the service does not actually start. running the status command confirms that the service is not running.

 

any ideas?

learning more.... removing the --quiet and -b option from the start-stop-daemon command in the init.d script, i now can produce an error output:

 

" Error while parsing config file: Invalid key 'jaked_file' "

 

My config.json file says jaked_file should be: 'home/pi/.btsync/data/.syncsystem/btsync.pid' which means I have found at least one error. The config.json file needs to auto-update that line to include the proper $BTSYNC_USER if the person installing this changes the user from 'pi'.

 

Manually changing this to '/home/jake/.btsync/data/.syncsystem/btsync.pid' still does not do the trick, unfortunately.

Got it. VERY unfortunately, it appears that whenever you wrote the script to swap out the default username 'pi' with the intended username, 'jake' in my case, all of the lines with 'pi' anywhere in them were swapped with 'jake'

 

that means all of the 'pid' variables turned into 'jaked' variables, which i had to change back manually.

 

maybe there is a cleaner way to write the script to swap out from the default user being pi?

 

regardless, thanks so much for writing this and putting it together! very helpful.

[Moe]

Thanks for reporting! I am working on a fix right now. That totally slipped through.. :/

[jake.sadie]

Just a follow-up for you. I have now installed this on two raspberry pis with non-default usernames and can confirm that taking these steps were necessary for the install to complete successfully:

• Edit btsync.sh to update to username I want to use to <user>
• Run btsync.sh to complete install
• Edit /etc/btsync/config.json
  • Line 7: change <user>d to pid
  • Line 7: change /home/pi/.btsync.... to /home/<user>/.btsync
  • Line 10: updated port number to reflect port number I was previously using (not necessary for everyone)
• Edit /etc/init.d/btsync
  • Line 21: change /home/pi/.btsync/data to home/<user>/.btsync/data
  • Line 25: change /home/pi/.btsync/data/.syncstream to /home/<user>/.btsync/data/.syncstream
  • Line 41: change db<user>d to dbpid
  • Line 42: change "$db<user>d" to "$dbpid"
  • Line 43: change "Stop<user>ng..." to "Stopping..."
  • Line 49: change db<user>d to dbpid
  • Line 50: change "$db<user>d" to "$dbpid"
  • Line 53: change ...(<user>d $db<user>d)" to ...(pid $dbpid)"

Again, not really a big deal to figure out, and thanks so much for doing this!

Just so you know, my primary use case is I have two raspberry pis with large HDDs hooked up, one on the west coast where I live and one on the east coast at my mom's place. I have both running as backup servers for my family so there are two places data is backed up. I normally run multiple accounts/processes on each, one per family member.

 

I previously wrote config scripts for each person, left them in the /etc/btsync folder, and the original init.d script back in the alpha/beta days would auto-load each whenever the process started. I'm not sure if this init.d script will do the same.

 

Might be a nice thing to add, though I am guessing it is likely a feature that will help such a smaller user base that you might want to omit it. Just a thought.

[Moe]

Alright, I looked at it and I fixed the regex used to replace the default user pi with the one a user wants to set.

 

The, now fixed, issue has been documented here.

 

You should not have to alter anything manual anymore. I checked the regex myself to verify it is working on all affected code pieces. If you may report back your findings, that would be perfect.

Also I would like to thank you for your feedback and detailed explanation of that issue - always helps!

 

So to have your issue fixed just get the updated btsync.sh script and you should be good!

curl -# -o btsync.sh https://raw.githubusercontent.com/moritzdietz/pi-syncscript/master/btsync.sh

[iswrong]

Just a tip, I see that you now use word boundaries to match 'pi' correctly. In general, people often use some extra markers to avoid ambiguity. E.g., you could put

 

%%%username%%%

 

in the configuration file and use sed to substitute all such instances. This has the added advantage that anyone reading the configuration file template can quickly see where substitutions will take place.

[Moe]

New release 1.1.1

 

Backup-ing

With this release I am overhauling the backup functionality.
Now I am doing it right. I am creating a tar ball that includes the following files and directories.

•     /home/pi/.btsync/
•     /etc/btsync/config.json
•     /usr/bin/btsync
•     /home/<user>/.btsync

        It does not include .log files or zipped .log files (sync.log.*.zip) nor does it include .journal files.

The .tar.gz file will be located in your home directory and is called btsync_backup_01.01.1970_00-00-00.tar.gz.
It is created using the tar -czPf flags using absolute paths. When you want to restore a backup it will restore all files listed above.

Version display

Now when you update your installation the script will output your current version and the one you just upgraded to.
For new installations it will also let you know what version you just installed.

Other than that I did some minor tweaks here and there and added/changed some of the verbose output for the end user.

 

You can get the latest version of the script following the instructions on the main page again.

[ThisIsNotYourDroid]

Thanks Moe!

 

I have Sync running on several Pi's and added another one this morning so decided to try out your script. It worked really well and I had Sync up and running under my Pro license within a few minutes. Like many users I attempted to run the script without fully reading then rereading the install instructions but knew right away where to look for the fix when the script bounced me the error "Could not create installation folder /home/pi/.btsync". I delete this account after setting up the Pi but the fix was quick and easy.

Cheers,

Billy

[Moe]

 You're welcome, glad it works so good for you

 

I am curious, how did you fix it? Did you use my instructions or did you do other changes?

[ThisIsNotYourDroid]

 You're welcome, glad it works so good for you

 

I am curious, how did you fix it? Did you use my instructions or did you do other changes?

The output from the script said it couldn't create items (sorry forgot what they were) in the pi user directory which I immediately saw. I figured it had to be something I could change in the script so I opened it and it was right there staring me in the face. I changed the pi user to my user name, reran the script and all is well. Then i went back to read your instructions and there it was again staring me in the face, "Prior to running this script you want to verify that the user variable on line 5 is correct."!   LOL.... glad to see that you thought of everything!

[ThisIsNotYourDroid]

Now that I have Sync running on two other Pi2's using this great script I thought I'd give it a go with OSMC on another Pi2. The script runs with no errors after updating line 5 in btsync.sh with the user as "osmc" and then setup autostart at boot with "sudo update-rc.d btsync defaults". After rebooting and checking "sudo service btsync status", all seems swell:

osmc@osmc:~$ sudo service btsync status● btsync.service - LSB: daemonized version of BiTorrent Sync.   Loaded: loaded (/etc/init.d/btsync)   Active: active (exited) since Mon 2015-11-30 19:54:23 EST; 12min ago  Process: 314 ExecStart=/etc/init.d/btsync start (code=exited, status=0/SUCCESS)Nov 30 19:54:22 osmc btsync[314]: Starting BitTorrent Sync for osmc

except all I get when accessing the webgui at <ip address>:8888/gui/ is "This webpage is not available". I scratched my head then checked the ports with "netstat -lptn" and I see that port 8888 isn't listed.

osmc@osmc:~$ netstat -lptn(Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.)Active Internet connections (only servers)Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program nametcp        0      0 0.0.0.0:111             0.0.0.0:*               LISTEN      -               tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      -               tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      -               tcp        0      0 0.0.0.0:1367            0.0.0.0:*               LISTEN      -               tcp        0      0 0.0.0.0:36666           0.0.0.0:*               LISTEN      -               tcp6       0      0 :::111                  :::*                    LISTEN      -               tcp6       0      0 :::80                   :::*                    LISTEN      -               tcp6       0      0 :::22                   :::*                    LISTEN      -               tcp6       0      0 :::36666                :::*                    LISTEN      -               tcp6       0      0 :::36667                :::*                    LISTEN      -               tcp6       0      0 ::1:9090                :::*                    LISTEN      -               

Checking the config file at "osmc@osmc:~$ sudo nano /etc/btsync/config.json" looks OK:

{  "vendor" : "Raspberry Pi",  "device_name": "Raspberry Pi",  "display_new_version": true,  "lan_encrypt_data" : false,  "storage_path" : "/home/osmc/.btsync/data/.syncsystem",  "pid_file" : "/home/osmc/.btsync/data/.syncsystem/btsync.pid",  "webui" :  {    "listen" : "0.0.0.0:8888",    "directory_root" : "/",    "directory_root_policy" : "all"  }}

I need to look deeper into OSMC but any ideas that someone may have would be appreciated. Since OSMC is based on Debian I thought this might be easy enough to install even though I had notice in other forums there were some issues.

 

Cheers!

[iswrong]

Now that I have Sync running on two other Pi2's using this great script I thought I'd give it a go with OSMC on another Pi2. The script runs with no errors after updating line 5 in btsync.sh with the user as "osmc" and then setup autostart at boot with "sudo update-rc.d btsync defaults". After rebooting and checking "sudo service btsync status", all seems swell:

osmc@osmc:~$ sudo service btsync status● btsync.service - LSB: daemonized version of BiTorrent Sync.   Loaded: loaded (/etc/init.d/btsync)   Active: active (exited) since Mon 2015-11-30 19:54:23 EST; 12min ago  Process: 314 ExecStart=/etc/init.d/btsync start (code=exited, status=0/SUCCESS)Nov 30 19:54:22 osmc btsync[314]: Starting BitTorrent Sync for osmc

except all I get when accessing the webgui at <ip address>:8888/gui/ is "This webpage is not available".

 

It's not running, it exited with status code 0 . You might want to try to get more log output through systemd:

 

sudo systemctl status -l btsync

 

or check the log file under

 

/home/osmc/.btsync/data/.syncsystem

 

to see why it exited.

[Moe]

You can also try to run with your current configuration

/usr/bin/btsync --nodaemon --config /etc/btsync/config.json

Or just the following to see if it will run at all

/usr/bin/btsync --nodaemon

Because if that runs there is something from with the init.d script.

[ThisIsNotYourDroid]

@iswrong & @Moe,

 

Thanks for the suggestions. I will check these and report back at the end of the week when I return from my biz travels. I mistakenly shutdown down this particular Pi before leaving so I can't access remotely. I don't know what I was thinking!

[ThisIsNotYourDroid]

Back from my travels.

 

@iswrong: thanks for pointing out that Sync hadn't started, I totally misread what the message said. I tried executing the "sudo systemctl status -l btsync" command and got the same returned message as running "sudo service btsync status" and the "/home/osmc/.btsync/data/.syncsystem" directory is empty.

 

@Moe: running either of your commands returned the below however btsync is present in the "usr/bin" directory.

osmc@osmc:~$ sudo /usr/bin/btsync --nodaemonsudo: unable to execute /usr/bin/btsync: No such file or directoryosmc@osmc:~$ sudo /usr/bin/btsync --nodaemon --config /etc/btsync/config.jsonsudo: unable to execute /usr/bin/btsync: No such file or directory

Before I had started this post I did some Googling and found a post in the OSMC forum regarding the need to link the below files which I had done under root but this hadn't worked at the time. This morning I tried linking these files again and after rebooting all is working now. I don't know why this didn't take the first time but stranger things have happened in my Pi adventures.

ln -s /lib/arm-linux-gnueabihf/ld-linux.so.3 /lib/ld-linux.so.3

Sync is now running (if not a little slow but that's for another time to investigate). I think you both for your help and will link Moe's Sync script in the OSMC forum post for others to try. This is going to be a great holiday present for some lucky person!

 

Cheers!

[erduan]

Hello,

 

I downloaded your script on a fresh Raspbian jessie installation and I'm encountering a problem I cannot fix.

 

After installing BTsync with your script (latest version 2.2.7), there is a bug at first.

It tells me to run BTsync with following command, which is not working:

"sudo service btsync start"

instead I had to start it with "sudo /etc/init.d/btsync"

 

Well, it's working and starting, I can open the webui over port 8888.

 

Now I wanted to change the port 8888 to 80, which worked on my manual installation:

"listen" : "0.0.0.0:8888", --> "listen" : "0.0.0.0:80",

 

The webui is just not starting anymore.

 

I looked at /etc/init.d/btsync and removed --quiet and -b of start(), which showed me details after running btsync.

 

It still shows port 8888 to me after running:

"BitTorrent Sync forked to background. pid = XYZ. default port = 8888"

 

Why doesn't the port change? I made the correct entry at the /etc/btsync/config.json file.

 

Edit: Same problem with all other ports than 8888

Edited January 1, 2016 by erduan

[Moe]

Please run the following command

/usr/bin/btsync --nodaemon --config /etc/btsync/config.json

and look for an error like this one

**** ERROR ******** Unable to listen for HTTP traffic on port 80.  Exiting...

You can see an overview using netstat to see to what ports your system is listening to using

sudo netstat -npl

I can confirm that using port 80, in your case, does not work.

This seems to be not a bug with the script, since the script starts BitTorrent Sync just fine, but something related to BitTorrent Sync itself (?!)

 

Because I can happily set the port to, for example, 8889 and it works like it should.

 

Also I don't know why you had to edit the init.d file? The deamon should work out of the box using the script.

[erduan]

Thanks for your reply.

 

I get the same error:

**** ERROR ******** Unable to listen for HTTP traffic on port 80.  Exiting...

But netstat shows me that port 80 is not in use.

pi@raspberrypi:~ $ sudo netstat -nplActive Internet connections (only servers)Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program nametcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      484/sshdtcp6       0      0 :::22                   :::*                    LISTEN      484/sshdudp        0      0 0.0.0.0:54702           0.0.0.0:*                           401/avahi-daemon: rudp        0      0 0.0.0.0:5353            0.0.0.0:*                           401/avahi-daemon: rudp        0      0 0.0.0.0:68              0.0.0.0:*                           483/dhcpcdudp        0      0 192.168.1.47:123        0.0.0.0:*                           500/ntpdudp        0      0 192.168.1.222:123       0.0.0.0:*                           500/ntpdudp        0      0 127.0.0.1:123           0.0.0.0:*                           500/ntpdudp        0      0 0.0.0.0:123             0.0.0.0:*                           500/ntpdudp6       0      0 :::5353                 :::*                                401/avahi-daemon: rudp6       0      0 :::44627                :::*                                401/avahi-daemon: rudp6       0      0 fe80::ba27:ebff:fed:123 :::*                                500/ntpdudp6       0      0 ::1:123                 :::*                                500/ntpdudp6       0      0 :::123                  :::*                                500/ntpdraw6       0      0 :::58                   :::*                    7           483/dhcpcdActive UNIX domain sockets (only servers)Proto RefCnt Flags       Type       State         I-Node   PID/Program name    Pathunix  2      [ ACC ]     STREAM     LISTENING     6671     1/init              /var/run/avahi-daemon/socketunix  2      [ ACC ]     STREAM     LISTENING     6673     1/init              /var/run/dbus/system_bus_socketunix  2      [ ACC ]     STREAM     LISTENING     6806     483/dhcpcd          /var/run/dhcpcd.sockunix  2      [ ACC ]     STREAM     LISTENING     6808     483/dhcpcd          /var/run/dhcpcd.unpriv.sockunix  2      [ ACC ]     STREAM     LISTENING     4000     1/init              /run/systemd/privateunix  2      [ ACC ]     SEQPACKET  LISTENING     4022     1/init              /run/udev/controlunix  2      [ ACC ]     STREAM     LISTENING     4026     1/init              /run/systemd/journal/stdout

I installed BTsync 2.2.7 manually before with the ARM package listed on getsync.com and I was able to use port 80 with BTsync, that's why I think it's related to the config or your init.d script.

 

Does someone have a clue?

 

Edit:

There seems to be the culprit!

Binding ports 1-1024 required root:

https://forums.freenas.org/index.php?threads/btsync-using-port-80-for-gui.22453/

 

Didn't knew that, I'm going to find a way around this.

Edited January 1, 2016 by erduan

[Moe]

Edit:

There seems to be the culprit!

Binding ports 1-1024 required root:

https://forums.freenas.org/index.php?threads/btsync-using-port-80-for-gui.22453/

 

Didn't knew that, I'm going to find a way around this.

 

Good to know! Thanks for the update

[erduan]

Added this to my iptables entries:

iptables -A PREROUTING -t nat -i eth0 -p tcp --dport 80 -j REDIRECT --to-port 8888

 

It's working now!

[Moe]

Just letting you guys know that the script works fine with the release of 2.3.

 

I am waiting for something to push an update out on the script. I will let you know when I have more info on that

[Moe]

I just released an update to the script!

Take a look at the release section of my GitHub - https://github.com/moritzdietz/pi-syncscript/releases/tag/v1.2.0