tuxpoldo

Debian And Ubuntu Server Unofficial Packages For Bittorrent Sync

Recommended Posts

By the way what i noticed during troubleshooting:

When you install btsync for the first time, the 'configurator' asks just a few settings.

After installing and you run dpkg-rconfigure, then it asks for all the settings.

Just wondering if thats normal behavior.

Share this post


Link to post
Share on other sites

By the way what i noticed during troubleshooting:

When you install btsync for the first time, the 'configurator' asks just a few settings.

After installing and you run dpkg-rconfigure, then it asks for all the settings.

Just wondering if thats normal behavior.

 

Yes - it is. During the first installation, the installer only ask the most important things in order to quickly get a running system. The options covered during the initial setup would probably satisfy 90%-95% of all installation requirements.

Share this post


Link to post
Share on other sites

Firstly, thank you so much for putting these packages together!

 

I've got sync up and running fine, but having issues with the web ui. Everything is running under a non-root user (called deploy). I'm using Ansible to deploy btsync so everything is automated.

 

The webui simply doesn't load, every browser reports that it can't connect (multiple browsers, networks, clear caches, etc). When I do a manual install and use the default config, the webui works fine. 

I've tried multiple ports and I'm definitely starting only one custom config I've made, so there are no port conflicts. Running with full logging (FFFF) gives me no errors. 

Is there a separate log for the webui? How should I debug this?

 

I'm running Ubuntu 12.04 on an AWS instance.

Edited by gamblor

Share this post


Link to post
Share on other sites

As everyone has said before, thank you for your work on the pkgs!

 

I am using for the past weeks btsync on a debian 7 machine. The only problem I am having is that I created the config by hand and not through dpkg-reconfigure. The reason being that dpkg doesn't allow me not to set a webui admin name / pass / port.

 

I have removed that part from the config and it works fine and all but everytime it updates it creates a similar default config (so I have two almost identical configs running at the same time). Could you make it so that if the user doesn't want the webui he can opt out of it during dpkg-reconfigure?

 

Thank you.

Share this post


Link to post
Share on other sites

Firstly, thank you so much for putting these packages together!

 

I've got sync up and running fine, but having issues with the web ui. Everything is running under a non-root user (called deploy). I'm using Ansible to deploy btsync so everything is automated.

 

The webui simply doesn't load, every browser reports that it can't connect (multiple browsers, networks, clear caches, etc). When I do a manual install and use the default config, the webui works fine. 

I've tried multiple ports and I'm definitely starting only one custom config I've made, so there are no port conflicts. Running with full logging (FFFF) gives me no errors. 

Is there a separate log for the webui? How should I debug this?

 

I'm running Ubuntu 12.04 on an AWS instance.

 

You should find out if anything runs as expected. Check the processes and the listen ports:

 

ps axu | grep btsyncnetstat -tlpe | grep btsync

 

These two commands show you

  1. which btsync processes are running and under which user credentials
  2. which user processes are listening under which ports

I'm quite sure, that you will find out, that the processes are not working as you think you configured them....

...

I have removed that part from the config and it works fine and all but everytime it updates it creates a similar default config (so I have two almost identical configs running at the same time). Could you make it so that if the user doesn't want the webui he can opt out of it during dpkg-reconfigure?

No, since it makes no sense (with a disabled web ui, debconf should be able to manage all folder shares). If you really want to have a very special custom configuration, with all shared directories defined in the configuration file, you have to do it manually. The reason for this, is that the configuration file method is not really deterministic (you can add an option, but if you remove it later, it stays still persistent since the configuration file is not the real configuration storage of btsync) and you have to understand the implications of working with a complex configuration file (and currently there is no architectural documentation available about this).

Share this post


Link to post
Share on other sites

Thanks so much for building, maintaining and supporting these packages. You're adding immense value to the official Linux binary release, and saving this husband-and-wife web development team significant time and trouble.

Share this post


Link to post
Share on other sites
 No, since it makes no sense (with a disabled web ui, debconf should be able to manage all folder shares). If you really want to have a very special custom configuration, with all shared directories defined in the configuration file, you have to do it manually.

 

 

 

I'm also running without the web ui (an unnecessary security risk IMO) and have thus answered "no" when asked by the installer to create a default config file. Instead I have custom config files in /etc/btsync. This seems to be working great, updating via apt-get stops and restarts my instances, all good.

 

AFAIK my config files have the default options, but the shared folders are listed explicitly in the config files.

 

 The reason for this, is that the configuration file method is not really deterministic (you can add an option, but if you remove it later, it stays still persistent since the configuration file is not the real configuration storage of btsync) and you have to understand the implications of working with a complex configuration file (and currently there is no architectural documentation available about this).

 

 

Ok I wasn't aware of this, and I'm not sure I understand it. What is the correct behavior when I e.g. add new shared folders, or change a configuration option in my config files? 

Share this post


Link to post
Share on other sites

....

Ok I wasn't aware of this, and I'm not sure I understand it. What is the correct behavior when I e.g. add new shared folders, or change a configuration option in my config files? 

 

The configuration file of BitTorrent Sync is a configuration initialisation facility and not a configuration storage facility. This creates a lot of misunderstandings. You can see it by trying the following:

 

In the web UI you can change a lot of settings that are also configurable through the configuration file (like username, password, ports, pnp settings, etc.). As soon as you change them from the web UI, they become effective. If you restart BitTorrent Sync all settings will be overwritten from the configuration file. If you do not want this, you have to remove the settings from the configuration file. In this case, they will be taken from the internal storage of BitTorrent Sync. This is really confusing for the user. Basically this refers also to the shared folders: create a configuration file with shared folders. After the first run, remove the shared folders from the configuration file and activate the web UI. You will see that the shared folders are still there (since saved in the internal database) and that you can manage them from the web UI.

 

The management of shared folders both from the web UI and from the configuration file would lead to an inconsistent representation, and for this reason the developers have disabled this possibility. (The only way to manage it in a consistent way would be to display the preconfigured folders from the configuration files in the web ui but to make then not manageable from there).

 

You can see a typical consequence of such misunderstandings here.

Share this post


Link to post
Share on other sites

Really sorry to spam your topic but I am new to BTSync, not new to Raspberry Pi. I have 1 RPi running XBian (XBMC media center based on Raspbian + installed some tools to get content I like). 

 

I now have a secondary RPi available, just installed Raspbian and I would like to run BTSync on this one. The RPi should be always on (will use it to sync photos from my mobile phone etc).

 

I understand I can simply install the official BTSync for RPi (using wget http://btsync.s3-website-us-east-1.amazonaws.com/btsync_arm.tar.gz), but I can also choose your package from this topic. Which one would be better? Also, why does Bittorrent still maintain their official release for this device if yours is better?

 

I would prefer not to depend on a 3rd party for future updates, would prefer to go for the official one. But it is all about performance eventually and after reading the 1st post your build seems the better choice.

Edited by zilexa

Share this post


Link to post
Share on other sites

I understand I can simply install the official BTSync for RPi (using wget http://btsync.s3-website-us-east-1.amazonaws.com/btsync_arm.tar.gz), but I can also choose your package from this topic. Which one would be better?

 

I can not tell you, which way of installing BitTorrent Sync would be better for you, since I have no idea, what are your requirements. This decision is yours, and nobody can take it from you. But there are a few things you should know:

  • The official distribution of BitTorrent Sync (the link above) contains the pure BitTorrent Sync executable and a text file with the license conditions. If you unpack the archive, you have to decide where to put the executable, how to manage startup and shutdown, how to create and maintain you configuration file, how to avoid multiple instances putting their internal data in different paths etc. etc....
  • The server or desktop packages for BitTorrent Sync provide a standard installation and operation scenario for BitTorrent Sync and wrap the original BitTorrent Sync executable in a Debian package that integrates the whole installation, maintenance and update procedure in the standard package management of your distribution. You will get a turnkey installation of BitTorrent Sync like you have on Windows and Mac OSX.
  • Differently from Windows and Mac OSX where BitTorrent Sync is distributed as a user application running in the context of a logged in user, the Linux version can also be operated as a server background task. If you want to run BitTorrent Sync as an application running for the logged in user, you should take a look at the Desktop Package. If you want BitTorrent Sync to run as a background task, the Server Package that is discussed in this thread might be the right one for you.

Also, why does Bittorrent still maintain their official release for this device if yours is better?

Who said, that "mine" is better?

 

BitTorrent Inc. releases the BitTorrent Sync executable because, they produce it and they want to support also the Linux platform. Without the release of the original BitTorrent executable also all derived distributions like the Server and Desktop Packages for BitTorrent Sync would not exist.

Share this post


Link to post
Share on other sites

Thanks for your explaination! I have read more pages now and finally understand this package. It seems like you have done a lot work I might have tried myself (and I would have failed definitely). I installed your package, will continue set up tomorrow.

What I want to do:

 

- make sure my phone and my gf phone can sync (backup) photos automatically to the RPi at home.

- have a sync of all my music on my RPi and 2 other RPis (3 in total, all different locations/cities) at other locations (I will install your package there as well)

- have a sync of 1 folder on a 4rd RPi on all other RPis --> reason why: I would like to have the data in this folder available wherever I am in the world and by having it on 4 RPis I hope, if I would ever need a file from that folder, the download speed will be fantastic because it gets its data from 4 sources.

 

I think your package is the way to go. Will experiment more this week. And also combine OwnCloud using info from here.

Share this post


Link to post
Share on other sites

[sOLVED: See Last Edit]

 

I have successfully followed the instructions for Debian Wheezy. However, I have run into some trouble.

I originally installed a 'default instance' using the supplied wizard. At that time, I choose to bind the web server to 0.0.0.0. I chose a very large password (64 characters). All worked well.

I changed the web server to bind to localhost.

`which sudo` dpkg-reconfigure btsync

Now, I decided that I don't need any password, because it is limited to localhost only. So, I ran the option to reconfigure the default instance; I removed the username (therefore allowing no authentication).

However, the problem is btsync still asks for (and accepts) the original username and 64 character password. In other words, it won't remove authentication...

Could this be looked into? Or am I just doing it wrong?

 

Here is my configuration file:

"device_name": "S1 - Default Instance","storage_path" : "/var/lib/btsync","listening_port" : 12354,"check_for_updates" : false,"use_upnp" : false,"download_limit" : 0,"upload_limit" : 0,"disk_low_priority" : true,"lan_encrypt_data" : true,"lan_use_tcp" : false,"rate_limit_local_peers" : false,"folder_rescan_interval" : 120,"webui" :{        "listen" : "127.0.0.1:4321"}

EDIT:

I found a solution. It seems I can remove authentication by using the webui itself (Preferences -> Authorization). When I remove the password here, it removes authentication successfully :D

 

Is this a bug? The reconfigure command says to leave the username blank to remove authentication. However, the webui says to leave the password blank to remove authentication.

 

It seems like a bug because the reconfigure command actually doesn't remove authentication (at least in my tests), whereas the webui does successfully remove authentication (although interestingly requires that the username is explicitly NOT blank).

 

Another bug, is that the secret cannot be copy and pasted (cannot be selected) on ipad (iOS) from the webui.

Share this post


Link to post
Share on other sites

Hello tuxpoldo,

thank you very much for your work, i found it very useful. I'm trying to implement a data sync service between two vps on linode, so when i found btsync, i discard all the other complex to manage solution that does not fit my needs. Right now i have a huge problem with btsync instances that are both running on its relative server, but can't see each other.

Both server running Ubuntu 13.10 and got same configuration (the second one is a clone of the first one). Since linode provide private networking, (and the private traffic is not billed :) ) i'm trying to perform sync over private network. I've installed bind-shim and set DAEMON_BIND to the respective private IP address in both machines. Now when is start/restart btsync service (DAEMON_INIT_DEBUG=1)  the message:

Bind address: '192.168.134.xx'

gives me hope about right interface has been binded for sync, but if i look at the /var/lib/btsync/sync.log:

[20140517 18:55:55.732] Using IP address xxx.xx.xxx.xx

tells me that btsync is using the public address instead the one i specified.

 

Here my /etc/network/interfaces as linode wants private address to be configured:

auto eth0 eth0:1#public addressiface eth0 inet static address xxx.xx.xxx.xx netmask 255.255.255.0 gateway xxx.xx.xxx.x#private addressiface eth0:1 inet static address 192.168.134.xx netmask 255.255.128.0

As you can see the private address is defined as secondary IP for eth0 using an interface alias. 

Can this be the problem? Googling around gives me on result, i'm the only one having this problem or maybe it's about a misconfiguration? Here my btsync conf file:

//// DAEMON_GID=root// DAEMON_BIND=192.168.134.xx//{        "device_name": "firstnode",        "storage_path" : "/var/lib/btsync",        "listening_port" : 0,        "check_for_updates" : false,        "use_upnp" : false,        "download_limit" : 0,        "upload_limit" : 0,        "disk_low_priority" : true,        "lan_encrypt_data" : false,        "lan_use_tcp" : true,        "rate_limit_local_peers" : false,        "folder_rescan_interval" : 600,        "send_buf_size" : 100,        "recv_buf_size" : 100,        "webui" :        {                "listen" : "127.0.0.1:8888",                "login" : "username",                "password" : "password"        }}

since webui listen to localhost, i use nginx as reverse proxy in order to access to the webui.

Hope you can help me in solve this problem. 

Best regards

Share this post


Link to post
Share on other sites

[sOLVED: See Last Edit]

...

EDIT:

I found a solution. It seems I can remove authentication by using the webui itself (Preferences -> Authorization). When I remove the password here, it removes authentication successfully :D

 

Is this a bug? The reconfigure command says to leave the username blank to remove authentication. However, the webui says to leave the password blank to remove authentication.

 

It seems like a bug because the reconfigure command actually doesn't remove authentication (at least in my tests), whereas the webui does successfully remove authentication (although interestingly requires that the username is explicitly NOT blank).

 

Another bug, is that the secret cannot be copy and pasted (cannot be selected) on ipad (iOS) from the webui.

Thank you for keep on trying (any finding the solution) ;-) That's exactly the solution that I would have suggested... As you noticed, no "login" and "password" parameter is specified in the configuration file. That's the way to configure a BitTorrent Sync without authentication according to the documentation. Unfortunately now you will feel a side effect of the architecture of BitTorrent Sync: the authorisation data is still there!

 

This is due to the fact that BitTorrent Sync is designed as a user application with a GUI. It has an internal data storage for all data structures. and when started for the first time, all parameters are initialised with decent default values.

 

Since the Linux Version has no real GUI, the concept is still valid, but they introduced also the configuration file in order to allow the user to specify his own defaults. If you make some experiments, you will see, that this will be the internal workflow...

  •  Initialise data structures with internal defaults
  • Read the config file and adapt data structure values with values specified in the config file
  • Start internal engine
    • If the storage_path exists and contains a valid database
      • Load databases (data structures will be overwritten)
      • Adjust all values based with the values specified in the config file (and if there are no credentials in the config file, the credentials from the database are still active)
  • else
    • First run!
  • Synching loop (normal program operation)
  • On Quit signal, write all internal data in databases in storage_path
  • end procedd
  • This is the big dilemma: The configuration file only sets the options specified inside. But since there is an internal storage, all other settings will stay persistent.

     

    Basically we can try the following:

     

    "webui" :        {                "listen" : "10.65.0.16:8888",                "login" : "",                "password" : ""        }
    Unfortunately the result is not as expected: BitTorrent continues to ask for credentials, but there are no valid credentials that permit to log on.

    The conclusion is, that your solution is the only valid one. Sorry. In any case I will report the problem also to the BitTorrent Inc. support. Perhaps they know a solution or implement a way to reset the credentials from the configuration file and then I will adapt my package.

Hello tuxpoldo,

thank you very much for your work, i found it very useful. I'm trying to implement a data sync service between two vps on linode, so when i found btsync, i discard all the other complex to manage solution that does not fit my needs. Right now i have a huge problem with btsync instances that are both running on its relative server, but can't see each other.

Both server running Ubuntu 13.10 and got same configuration (the second one is a clone of the first one). Since linode provide private networking, (and the private traffic is not billed :) ) i'm trying to perform sync over private network. I've installed bind-shim and set DAEMON_BIND to the respective private IP address in both machines. Now when is start/restart btsync service (DAEMON_INIT_DEBUG=1)  the message:

Bind address: '192.168.134.xx'
gives me hope about right interface has been binded for sync, but if i look at the /var/lib/btsync/sync.log:

[20140517 18:55:55.732] Using IP address xxx.xx.xxx.xx
....
 

 

Hi panza!

 

Please do not give too much on the log file: the SHIM is an external workaround and BitTorrent Sync does not know anything about it. You can verify the binding of the interface by issuing the command:

 

yeasoft-gate2 ~ # netstat -tulpe | grep btsync
Without shim you will get something like this:

 

tcp        0      0 *:26185                 *:*                     LISTEN      root       3798794     29900/btsync-daemontcp        0      0 yeasoft-gate2.ffm.:8888 *:*                     LISTEN      root       3798807     29900/btsync-daemontcp        0      0 *:22144                 *:*                     LISTEN      root       3798823     29989/btsync-daemonudp        0      0 *:26185                 *:*                                 root       3798805     29900/btsync-daemonudp        0      0 *:22144                 *:*                                 root       3798834     29989/btsync-daemonudp        0      0 *:3838                  *:*                                 root       3802432     29989/btsync-daemonudp        0      0 *:3838                  *:*                                 root       3803292     29900/btsync-daemon
With an active shim, you will see this:

 

linux-dev-64 ~ # netstat -tulpen | grep btsynctcp        0      0 127.0.0.1:8888          0.0.0.0:*               LISTEN      0          11836       2400/btsync-daemontcp        0      0 10.65.6.31:64127        0.0.0.0:*               LISTEN      0          11823       2400/btsync-daemonudp        0      0 10.65.6.31:64127        0.0.0.0:*                           0          11834       2400/btsync-daemonudp        0      0 10.65.6.31:3838         0.0.0.0:*                           0          10755       2400/btsync-daemon
And here you can see that btsync is only bound to the internal interface. Remember that btsync works fine also on system connected only to internal interfaces: it determines the external address by communicating with the external relay/tracker servers. If you want btsync to limit the communication only to internal systems or systems with known and reachable addresses, you have to disable some options in the folder share settings ("Use tracker server" and "Search DHT network") and add the addresses of all peers involved in the sync (in case they are not on the same LAN segment).

Here you can see a real life example of a hardened btsync configuration for synchronising the PXE boot areas across all the main routers in my internal infrastructure.

//!/usr/lib/btsync/btsync-daemon --nodaemon --config//// This btsync instance provides sync services for// all common files in the YeaSoft Server Grid//// DAEMON_NICE=10//{	"device_name" : "yeasoft-gate2 - Server Sync",	"storage_path" : "/var/lib/btsync-serversync",	"listening_port" : 22144,	"check_for_updates" : false,	"use_upnp" : false,	"download_limit" : 0,	"upload_limit" : 0,	"lan_encrypt_data": true,	"lan_use_tcp" : true,	"rate_limit_local_peers" : false,	"webui" : {	}	,	"shared_folders" : [ {		"secret" : "<removed>",		"dir" : "/mnt/data/tftproot/yeaboot",		"use_relay_server" : false,		"use_tracker" : false,		"use_dht" : false,		"search_lan" : true,		"use_sync_trash" : true,		"known_hosts" : [			"10.65.4.1:22144",			"10.65.6.1:22144",			"10.65.8.1:22144",			"10.65.16.1:22144",			"10.65.18.1:22144"		]	}	]}

Share this post


Link to post
Share on other sites

Unfortunately yes. The btsyncindicator.py from Mark Johnson is not working with the new version (since the web UI has changes so much) and this would break btsync-user. I'm still waiting for an update... But perhaps I will release today with a limitation in the version dependencies....

Reported in the btsyncindicator project as Issue #50

I hope we will get an update soon...


I added in the package a version limitation ( Breaks: btsync-user (<= 1.15.0-1) ) and this causes btsync-common to be kept back on systems with btsync-user installed. I think that I can now upload the update...
 

leo@debpoldo /etc/apt/sources.list.d $ sudo apt-get upgradeReading package lists... DoneBuilding dependency tree       Reading state information... DoneThe following packages have been kept back:  btsync-commonThe following packages will be upgraded:  google-chrome-stable libssl1.0.0 libxfont1 linux-headers-3.2.0-4-686-pae linux-headers-3.2.0-4-common  linux-image-3.2.0-4-686-pae linux-libc-dev openssl tzdata tzdata-java yea-base-files11 upgraded, 0 newly installed, 0 to remove and 1 not upgraded.Need to get 79.1 MB of archives.After this operation, 2,003 kB disk space will be freed.Do you want to continue [Y/n]? 

Share this post


Link to post
Share on other sites

I have the desktop packages and it says in the about:  Version 1.3.105 (1.3.105) - Linux UI Version 0.8.3

 

But it seems to work fine on Mint 17.

Share this post


Link to post
Share on other sites

I have the desktop packages and it says in the about:  Version 1.3.105 (1.3.105) - Linux UI Version 0.8.3

 

But it seems to work fine on Mint 17.

 

You are using the next generation desktop package (btsync-gui). The problem is related to the legacy desktop package (btsync-user) that is still supported....

Share this post


Link to post
Share on other sites

Hi, there is a translation problem for italian language. Could you fix it? Thank you.

g4gt.png

 

Sorry, but you are in the wrong thread. The Web UI is part of the core BitTorrent Sync code, therefore you should report this in a separate topic. 

Share this post


Link to post
Share on other sites

You're right, I'm sorry.  :unsure:

 

No problem! My concern is only that your report is not read by the right people.

Share this post


Link to post
Share on other sites
Guest
This topic is now closed to further replies.