Debian And Ubuntu Server Unofficial Packages For Bittorrent Sync


Recommended Posts

BTSYNC PACKAGES FOR DEBIAN, UBUNTU AND OTHER DERIVED DISTRIBUTIONS

BitTorrent® Inc. delivers for Linux users only a raw binary file without any deployment concept or setup system. It's up to the user to create a reliable startup and shutdown logic, manage configuration files, internal storage directories and everything else related to file and directory permissions, application update and various other aspects.

The BitTorrent Sync Server Package (btsync) is the ideal solution for all users that want to deploy BitTorrent Sync on Linux servers running Debian or other derived distributions like Ubuntu, Raspbian, Linux Mint or similar.

THIS SERVER PACKAGE IS UNOFFICIAL AND NOT THE WORK OF BITTORRENT® INC. PLEASE DO NOT CONTACT THE BITTORRENT® INC. SUPPORT WITH QUESTIONS OR PROBLEMS RELATED TO THE USE OF THE PACKAGE. YOU WILL FIND COMPETENT HELP AND SUPPORT IN THIS THREAD

TYPICAL USE CASES

The BitTorrent Sync Server Package is designed to run one or more BitTorrent Sync background processes (called "instances") on servers where no specific user is usually logged on. Since the server package does not provide any GUI (except for the optional Web UI provided by BitTorrent Sync itself), it can be also installed on headless servers (without any desktop environment).

The server version is particularly suitable for the following use cases:

  • BitTorrent Sync is used as a background service, to keep directories in sync between all servers of a distributed infrastructure like a PXE boot system and groups of shared configuration files.
  • Always-on instances of BitTorrent Sync for providing an always available external copy and source of replicated data for other BitTorrent Sync clients.
  • A content distribution network based on shared folders.
  • Customized services built upon the functionality of BitTorrent Sync.
This topic covers only the server package. If you are searching for the desktop user package, please look here.

INSTALLATION ON DEBIAN, UBUNTU, LINUX MINT, RASPBIAN OR OTHER DEBIAN DERIVATES

The most easy and fast way to install the repository with latest stuff is to paste that at a terminal prompt:

sh -c "$(curl -fsSL http://debian.yeasoft.net/add-btsync-repository.sh)"
If instead you wish to stay on version 1.4, you should paste the following command at a terminal prompt:

sh -c "$(curl -fsSL http://debian.yeasoft.net/add-btsync14-repository.sh)"
The script explains what it will do and then pauses before it does it asking for your permission. If you encounter any problems or prefer to do it manually, please look here.

Now update the package index and install btsync by pasting that at a terminal prompt:

`which sudo` apt-get update`which sudo` apt-get install btsync
After downloading the packages, the installation begins. The package manager will ask you, if you want a default instance of BitTorrent Sync to be created. If you answer yes, you will be guided through the installation and when finished you will have a fully operational BitTorrent Sync instance maintained by debconf. The configuration can be modified and fine tuned at any time by performing the following command:

`which sudo` dpkg-reconfigure btsync
Here you can also choose to delete the default instance by answering no to the initial question.

USAGE NOTES

The BitTorrent Sync Server Package mainly consists of an init-style startup script that manages every operational aspect of running BitTorrent Sync instances.

Instance Configuration Files

All operational parameters for BitTorrent Sync instances are configured in configuration files located in the directory /etc/btsync. These configuration files basically are JSON Files and follow a very strict syntax. In addition to the JSON specification, comments are also supported. The support of commented lines permits to specify additional parameters that are not directly parsed by the BitTorrent Sync executable itself but by the init-script.

The naming convention for configuration files defining BitTorrent Sync instances is:

<instance_name>.conf
where <instance_name> should be a string consisting of alphanumeric characters.

The default instance maintained by debconf has also a configuration file that is named debconf-default.conf. This name shall never be used by manually created configuration files, since it may be deleted or overwritten by debconf. This file shall also never be edited manually since it is often rewritten and/or created/deleted by debconf. If you want to manage the default instance, you must always use the command:

`which sudo` dpkg-reconfigure btsync
Each configuration file in /etc/btsync defines a separate running BitTorrent Sync instance (that means: a process). When defining more than one instance on a system, there are some rules that must be considered:
  • The listening ports (parameter listening_port) must be unique for each BitTorrent Sync instance
  • The internal database path (parameter storage_path) must be unique for each BitTorrent Sync instance
  • If the Web UI is enabled, the Web UI bind address and listening port (parameter listen) must be unique for each BitTorrent Sync instance
The syntax and configuration parameters of a BitTorrent Sync configuration file are partially documented in the example file /etc/btsync/samples/complex.conf and in the official BitTorrent Sync manual which can be also found in /usr/share/doc/btsync-common/BitTorrentSyncUserGuide.pdf.gz.

There are additional parameters that are parsed by the init script upon startup and affect many important aspects. These parameters are not parsed directly by BitTorrent Sync and therefore have to be placed in comment lines. In order to improve the readability of configuration files, they should be grouped together and placed at the beginning of the configuration file. The generic syntax of such parameters is:

// PARAMETER_NAME=<parameter value>The following parameters are supported:
  • DAEMON_UID: A uid (user id) for specifying the user under which the BitTorrent Sync instance should run
  • DAEMON_GID: A gid (group id) for specifying the group under which the BitTorrent Sync instance should run
  • DAEMON_UMASK: The umask (up to 4 octal digits) for the BitTorrent Sync instance. If omitted the default umask is used.
  • DAEMON_DEBUG: The debug mask (4 hex digits) for the BitTorrent Sync instance. If omitted the init script will not touch potential settings defined manually. If set to 0000, a potential settings file will be deleted. Full detail is set by specifying FFFF. The log file can be found in the directory specified by storage_path and is named sync.log
  • DAEMON_NICE: The niceness level of the BitTorrent Sync instance, which affects the process scheduling. Nicenesses range from -20 (most favorable scheduling) to 19 (least favorable).
  • DAEMON_BIND: This parameter works only in conjunction with the package bind-shim. Please see the specific section below covering the specific interface binding topic.
//// DAEMON_UID=jdoe// DAEMON_UMASK=0002//
This example will launch the instance running under the credentials of the user "jdoe" using 0002 as umask.

By specifying DAEMON_UID and DAEMON_GID it is possible to specify under which credentials the instance runs. In this case it must be assured that the instance is able to read and write all files it must access:

  • The instance must be able to read its own configuration file. Since the configuration file may contain sensitive data (the userame and password for the web interface), it is usually owned by the user under which the instance runs and has the mode 400 (-r--------) in order to limit the access to the specific user. Usually the init script takes care of adjusting the file permissions.
  • The instance must be able to read and write in its storage_path, since all internal control and status files (and the database of file hashes) are kept there. The ideal solution is to keep the storage_path somewhere in the home directory of the user under which the instance will run. Usually the init script takes care of adjusting the file permissions.
  • Obviously the instance must be able to read and write the files and directories that will be synchronized.
Some example configuration files are provided under /etc/btsync/samples

Modular Configuration Files

Sometimes it may be useful to generate configuration files out several separate parts. The typical use case is explained best by forum user Jero:

Is it possible to split the linux config file in multiple files?

 

I want to have a config file per shared folder. I currently manage 10-20 linux servers. The servers are all connected with btsync.

I have created a system that can push scripts to the servers en executed them. But when i need to change one shared folder on one server i just want to push a new file without to touch the other parameters in the config file.

If you want to create a modular configuration file, create a directory in /etc/btsync named like you would name the configuration file, but with an additional .d extension and put the parts of your configuration file in it. The daemon init script will generate a configuration file at every start by joining the parts in alphabetical order. Parts must be text files with the extension .part.

Default Startup Parameters

The additional configuration file /etc/default/btsync permits to specify some default startup parameters that may affect every configured instance of BitTorrent Sync.

The variable AUTOSTART defines which instances are started automatically. It can take the following values:

  • none No instance is started automatically
  • all (default) All instances are started automatically
  • <list> Only the specified instances are started automatically
Instances not started automatically upon system startup can be managed manually from the command line:

# start a specific instanceservice btsync start <instance_name># stop a specific instanceservice btsync stop <instance_name>
The advanced variable DAEMON_ARGS allows to specify additional command line parameters passed to the daemon. WARNING: Be careful! You should only add things here if you know EXACTLY what you are doing!

The variable DAEMON_BIND has the same purpose as the same parameter in the instance configuration files, but in this scope it will be applied by default to all instances. Please see the specific section below covering the specific interface binding topic.

The variable DAEMON_INIT_DEBUG permits to enable extended debug output of the init-script.

HANDLING OF SSL CERTIFICATES

BitTorrent Sync 1.4 supports also SSL connections to the Web UI. The default instance is automatically configured to use a self signed certificate and key created during the installation.

The configuration file does not point directly to the self signed certificate but to the following symbolic links:

/etc/btsync/debconf-default.crt/etc/btsync/debconf-default.key
The default instance can be operated also with user provided certificates without tampering with the configuration file by deleting the symbolic links and replacing them with files containing the desired certificate and key. Debconf will detect that the files are not symbolic links and leave them untouched during reconfiguration.

The certificate file and key file must be accessible by the BitTorrent Sync daemon.

BINDING BITTORRENT SYNC TO A SPECIFIC INTERFACE

Unfortunately BitTorrent Sync currently does not support to bind the main service routine to a specific network interface. Currently only the internal web server providing the Web UI and the BitTorrent Sync API can be specifically bound to a specific address.

In order to limit the operation of BitTorrent Sync to a specific interface in a multihomed environment, a so called preload shim) can be used. The BitTorrent Sync repository contains a precompiled version of Daniel Ryde's bind.so shim that must be installed with:

`which sudo` apt-get install bind-shim
The BitTorrent Sync init script does detect the presence of the library and enables support for the additional parameter DAEMON_BIND that can be specified either in /etc/default/btsync or in instance configuration files.

By specifying a valid TCP/IP address assigned to one of the interfaces of the system, the BitTorrent Sync instance will bind only to that specific address. This will affect also the Web UI, if 0.0.0.0 is specified as the bind address.

COMPATIBILITY

The server packages are available for the same architectures as released by BitTorrent Inc.:

  • i386 Intel/AMD 32 Bit
  • amd64 Intel/AMD 64 Bit
  • armel ARM EABI
  • armhf ARM hard float
  • powerpc PowerPC
Since the packages still have not been tested on every platform, any related feedback is highly appreciated.

BUG REPORTS, CONTRIBUTION AND SOURCES

If you want to contribute to the development of the packages or if you are curious how this all works, you may find the current sources of the deployment scripts and packaging on GitHub under the following URL: https://github.com/tuxpoldo/btsync-deb

If you have experienced a reproducible issue that is related to the packaging and not to BitTorrent Sync itself you are strongly encouraged to open an issue on the project's GitHub Page. Issues related to the native functionality of BitTorrent Sync should rather by creating a new topic or partecipating to an already existing topic in the BitTorrent Sync Forum.

Edited by tuxpoldo
Link to comment
Share on other sites

there's already a newer version than the one on PPA.

Thank you for the information but: is it an official version? I'm uncertain if I should create a new package version using that binaries, since it seems not to be released over the normal download links or the autoupdate mechanism.

Hey staff! Any information about that?

Link to comment
Share on other sites

The latest official "public" build on the website is 1.0.116.

1.0.125, however, has since be made available through the forums to users experiencing issues with 1.0.116, but 1.0.125 is considered a "test build" (mind you, it could be argued that all these builds are "test builds" anyway as we're still in "alpha"!)

So, tuxpoldo, 1.0.125 is endorsed by BitTorrent Sync, as it's been posted a number of times in the forums now by BitTorrent staff (for instance here, here, here and here)... but, that said, I wouldn't spend too much time packaging up 1.0.125, as there is anticipated to be a more significant update made generally available this next week! ;)

Link to comment
Share on other sites

Unpacking btsync (from btsync_1.0.125-1_amd64.deb) ...
dpkg: dependency problems prevent configuration of btsync:
btsync depends on libc6 (>> 2.15); however:
Version of libc6:amd64 on system is 2.13-38.

dpkg: error processing btsync (--install):
dependency problems - leaving unconfigured
Errors were encountered while processing:
btsync

^<- didn't work on my Debian.

http://syncapp.bittorrent.com/1.0.130/ is out now btw, the http://syncapp.bittorrent.com/1.0.130/btsync_x64-1.0.130.tar.gz build there did work for me

Link to comment
Share on other sites

He wrote some funky setup scripts (dpkg-configuration) that ask you to enter some initial settings (so you don't necessarily have to write your own JSON). His default binary path was "/usr/sbin/btsync-daemon" on my system, rather than for example keeping the binary named "btsync".

His /etc/init.d/ scripts didn't work for me, don't know if that's because the setup ultimately failed for me or because of something being wrong with his script or....

His launch script also makes sure you have "check_for_updates" : false, else it refuses to start.

Link to comment
Share on other sites

Hi!

His /etc/init.d/ scripts didn't work for me, don't know if that's because the setup ultimately failed for me or because of something being wrong with his script or....

It would be really helpful to have some more information about what's the problem. Which OS are you running? Do you have installed with the DEB packages or do you have only taken the scripts in order to make your own deployment?

His launch script also makes sure you have "check_for_updates" : false, else it refuses to start.

The reason for that is simple: if you have software managed by the package management of an operating system, it's the package management that manages the updates ;-)

Link to comment
Share on other sites

I wrote the error I got trying to install your .deb.

I'm on Debian Unstable. x64 of course.

I renamed the .dpkg-new init.d scripts to use them with the working build from the .tar.gz (which I copied into your /usr/sbin/btsync-daemon location).

Link to comment
Share on other sites

I wrote the error I got trying to install your .deb.

I'm on Debian Unstable. x64 of course.

I renamed the .dpkg-new init.d scripts to use them with the working build from the .tar.gz (which I copied into your /usr/sbin/btsync-daemon location).

Thank you for the information! I will install a machine with debian unstable x64 and try to fix the problem.

Link to comment
Share on other sites

The bug in the debian versions have been fixed and it's now possible to install them on debian squeeze, wheezy and sid. There was a dependency problem, that caused the package to throw an error during installation.

Starting from now, there will be also a repository available for debian. See the initial posting for instructions on activating it on your machine.

Link to comment
Share on other sites

Installing via the package indeed works now :)

What does not work for me is the "<name>.<user>.{<group>.}conf" type of config. It just refuses to start as a user and not as root. If I rename the same config to "<name>.conf", it starts fine via your script.

Manually launching BTSync as my desired user also works.

Link to comment
Share on other sites

That's strange. I've tested it on my Debian sid amd64 installation and it works. The only problem I encountered was some syntax errors in /etc/btsync/samples/user.jdoe.conf

If you take that file, and simply rename it for your user name and change some settings inside, it will not work, since there are internal syntax errors... BTW: I check the sample files in the next release.

In any case, please try the following in a shell running under the user account of the user as which you want to run the daemon:


/usr/sbin/btsync-daemon --nodaemon --config /etc/btsync/<filename>.conf

Perhaps we will see, what's the problem....

BTW: What are the permissions of the configuration file? If you have a password (for the web ui) stored inside, it makes sense to have permissions like 400, but the problem is that the daemon will not be able to read the file. In that case I would suggest to change also the owner of the configuration file to the user...

Link to comment
Share on other sites

Ah, my user did not have permission to write the .pid file. Now everything works, cheers :)

And yes, I noticed the JSON errors in the sample config myself when I was first testing it. Thanks to that I now know JSON syntax again, hadn't used it in a while.

By the way, while it's very forthcoming of you to also send me your replies via PM, you can see in the top right corner of this board's design that I'm subscribed to this thread :)

Link to comment
Share on other sites

Dear Tuxpoldo

Thank you to provide us this packages. I ran into the same probleme as lux

I use ubuntu, I change the file in /ect/btsync to config.gilles.gilles.conf

When I run : /etc/init.d/btsync start I get

== START ===========================

File Name: 'config.gilles.gilles.conf'

Name Part: 'config'

Credential Part: 'gilles:gilles'

Run As User: 'gilles'

Run As Group: 'gilles'

unfortunatelly btsync stils run under root

946 root 20 0 379m 4036 2144 S 1.0 0.2 1:32.49 btsync-daemon

Do you have any idea?

By the way shouldn't we run btsync with a new user btsync:btsync? Then we could add basic user to the btsync group. This would be safer than using root and allow a basic user to access the files.

## OK I found the probleme, same solution as LUX worked

Link to comment
Share on other sites

Hi Gilles!

## OK I found the probleme, same solution as LUX worked

I'm glad to head about that. I saw that you uncommented my debug output in start_btsync () - right approach to solving problems! I'm still wondering about that problems, but I think that the main cause is, that I operate linux only as server and I work mainly on mac workstations. I will install a full desktop linux virtual machine in order to test this stuff from a user perspective...

By the way shouldn't we run btsync with a new user btsync:btsync? Then we could add basic user to the btsync group. This would be safer than using root and allow a basic user to access the files.

This is definitively an approach that every user can deploy, if he likes: create the user btsync, define the configuration file in the form myconfig.btsync.btsync.conf and then you have exactly what you want. But unfortunately this does not solve the main problem: btsync is not able:

  1. to preserve file ownership through the replication process
  2. to preserve all file attributes (ACL and permissions)

btsync is designed as a single user tool, that has to be executed in the context of a logged in user in a typical desktop session. My approach to deploy it as a system service is more intended to satisfy the following two fundamental use cases:

  1. Realtime replication of internal server content in a server farm. e.g.: I use btsync to synchronise common configuration information, important documents and the PXE boot system area (about 10 GB) across all sites of my virtual company locations (Locations: data center, my residence in Munich, residence of my parents in Lucca, and two residences of connected friends that share my infrastructure). In this case running as root is OK, since all this stuff is stored as root:root with permissions 0x644
  2. Realtime replication of some precious personal content (the one that is managed on the desktop computers with the desktop versions of btsync) on a central location in the data center. That is - simply spoken - the "cloud storage feature" of btsync ;-) These instances run also on the data center servers under the credentials of their respective users. This includes also instances running on my NAS devices at home.

Link to comment
Share on other sites

Thank you for your advice.

To my understanding there is two usages to btsync

One for desktops and one for servers

For server your package is perfect (maybe a special user by default would be more secure). It will allow me to get my backups out of our datacenter to our office.

For desktop we could have another package or an installer so that the btsync is automatically configured to run when the session starts and to avoid having the binary into download folder or on the desktop (like I did when I first used it). Well off course I could run a Mac ;)

Link to comment
Share on other sites

I think, you are right. For Desktop usage, it should work the same way, it works for Windows and Mac OSX. The application is installed somewhere with autostart, a default configuration file is placed in the home directory of the user and in the /etc/skel directory and a menu entry is created for opening the web ui in the browser. I will start next week developing also such a package.

Link to comment
Share on other sites

Guest
This topic is now closed to further replies.