Sign in to follow this  
no1youknowz

Sync Takes Up To 10 Minutes On Change After Initial Startup

Recommended Posts

Hi,

 

I generated my own rpm for centos using the latest source and spec file here: 

https://github.com/funzoneq/rpmspecs/tree/master/btsync

I'm using 2 centos 6.5 servers on the same lan.  Server A (acts as a server) and has the secret generated and folder shared.  Server B (acts as a client) and has a read-only folder of Server A.

 

Upon start of btsync on Server B, I see in the logs of Server A:

[20140711 18:06:01.521] Incoming connection from 10.0.4.9:9999

All the files on Server B are accounted for, so I know the connectivity between both servers work and I see in the web UI for Server A that Server B has indeed connected.

 

The problem now, is that whenever I do a change on Server A.  I have to wait at least 10 minutes sometimes for the sync to occur on Server B.  That is an addition of a new file, deletion of a file or modification of a file.

 

Here are the respective configs.

 

In addition, if anyone can advise also on what ports I need open as I will be moving this to AWS and would like to secure the respective security groups in my VPC.

 

Thanks

 

Server A:

{  "device_name": "Platform",  "listening_port" : 9999,                       // 0 - randomize port/* storage_path dir contains auxilliary app files   if no storage_path field: .sync dir created in the directory   where binary is located.   otherwise user-defined directory will be used*/  "storage_path" : "/root/.sync",// uncomment next line if you want to set location of pid file// "pid_file" : "/var/run/btsync/btsync.pid",  "check_for_updates" : true,  "use_upnp" : false,                              // use UPnP for port mapping/* limits in kB/s   0 - no limit*/  "download_limit" : 0,  "upload_limit" : 0,/* remove "listen" field to disable WebUI   remove "login" and "password" fields to disable credentials check*/  "webui" :  {    "listen" : "0.0.0.0:8888"  }// Advanced preferences can be added to config file.// Info is available in BitTorrent Sync User Guide.}

In addition in the UI I have these set:

 

"use_relay_server" : false,
"use_tracker" : false,
"use_dht" : false,
"search_lan" : false,

 

Server B:

{  "device_name": "web1",  "listening_port" : 9999,                       // 0 - randomize port/* storage_path dir contains auxilliary app files   if no storage_path field: .sync dir created in the directory   where binary is located.   otherwise user-defined directory will be used*/  "storage_path" : "/root/.sync",// uncomment next line if you want to set location of pid file// "pid_file" : "/var/run/btsync/btsync.pid",  "check_for_updates" : true,  "use_upnp" : false,                              // use UPnP for port mapping  "shared_folders" :  [    {//  use --generate-secret in command line to create new secret      "secret" : "ABCD",                   // * required field      "dir" : "/srv/www/www.domainA.com/public_html", // * required field//  use relay server when direct connection fails      "use_relay_server" : true,      "use_tracker" : true,      "use_dht" : false,      "search_lan" : true,//  enable SyncArchive to store files deleted on remote devices      "use_sync_trash" : true,//  specify hosts to attempt connection without additional search      "known_hosts" :      [        "10.0.4.2:9999"      ]    },    {//  use --generate-secret in command line to create new secret      "secret" : "ZYXW",                   // * required field      "dir" : "/srv/www/www.domainB.com/public_html", // * required field//  use relay server when direct connection fails      "use_relay_server" : true,      "use_tracker" : true,      "use_dht" : false,      "search_lan" : true,//  enable SyncArchive to store files deleted on remote devices      "use_sync_trash" : true,//  specify hosts to attempt connection without additional search      "known_hosts" :      [        "10.0.4.2:9999"      ]    }  ]// Advanced preferences can be added to config file.// Info is available in BitTorrent Sync User Guide.}

Share this post


Link to post
Share on other sites

Try decreasing the "folder_rescan_interval" setting.

 

This controls how frequently Sync re-scans folders for changes that can't be picked up in real-time. By default, this is set to 600 seconds i.e. 10 minutes.

Share this post


Link to post
Share on other sites

Try decreasing the "folder_rescan_interval" setting.

 

This controls how frequently Sync re-scans folders for changes that can't be picked up in real-time. By default, this is set to 600 seconds i.e. 10 minutes.

 

Thanks for replying.  I really appreciate it!!

 

I tried this and it really makes no difference.

 

I enabled debugging and here is what I found:

 

Server B:

140711 19:30:06.078] ping 10.0.4.2:47912[20140711 19:30:06.079] Checking connection to 10.0.4.2:47912:TCP[20140711 19:30:06.079] Send ping to peer (1051679F8A3AF0150261CDC895BB42110345E8BE) for share 1866700239B4DD73623256D3B6F722E5022F968B:[20140711 19:30:06.079] ping 10.0.4.2:47913[20140711 19:30:06.079] Checking connection to 10.0.4.2:47913:TCP[20140711 19:30:06.079] Send ping to peer (0000000000000000000000000000000000000000) for share 1866700239B4DD73623256D3B6F722E5022F968B:[20140711 19:30:06.079] ping 10.0.4.2:9999[20140711 19:30:06.079] Got ping (broadcast: 0) from peer 10.0.4.2:9999 (1051679F8A3AF0150261CDC895BB42110345E8BE) for share 6637CB8E02DA15848330FC2230FF53F4545C9BE1[20140711 19:30:06.079] Got ping (broadcast: 0) from peer 10.0.4.2:9999 (1051679F8A3AF0150261CDC895BB42110345E8BE) for share 1866700239B4DD73623256D3B6F722E5022F968B

Looks good right?

 

Then I get:

[20140711 19:03:48.364] Failed to open tunnel to 10.0.4.2:50985:TCP[20140711 19:03:48.364] Failed to open tunnel to 10.0.4.2:50987:TCP[20140711 19:03:49.230] Lost tunnel to 10.0.4.2:9999:uTP[20140711 19:03:49.230] Lost tunnel to 10.0.4.2:9999:uTP[20140711 19:03:49.230] Lost tunnel to 10.0.4.2:9999:uTP[20140711 19:03:49.230] Lost tunnel to 10.0.4.2:9999:uTP

I see this on the other end.  It starts pinging and getting the tunnels ok.  But then fails!

 

Also, is there a way of deleting devices?

 

I keep on seeing:

[20140711 19:03:47.429] Send ping to peer (103884B3708AADEE2787F1445C69066B816AF1F6) for share 6637CB8E02DA15848330FC2230FF53F4545C9BE1:[20140711 19:03:47.429] ping 10.0.4.232:9999

When this device no longer exists.  It would be nice to tell Btsync to remove devices from its list when it hasn't heard from it in 5 mins or so.

Here is some more information....

--------------------

 

 

 

 

 

 

 

Thanks

 

Why is this failing so much?

 

I do an update with a file.  Like creation or deletion...

 

 

[20140711 19:43:42.369] Incoming connection from 10.0.4.2:47973

[20140711 19:43:42.372] SyncFolder::ProcessRequest() received "state_notify" request.

[20140711 19:43:42.372] Got state notify from peer 0000000000000000000000000000000000000000 - tree:979EBC63C701130BDCBF656A9F03778DE1C42AED pieces:B56613ABCD5DFDFB9AFD2DB210A29A8FB9D9859F

[20140711 19:43:53.090] Failed to open tunnel to 10.0.4.2:9999:TCP

[20140711 19:43:53.090] Failed to open tunnel to 10.0.4.2:9999:TCP

[20140711 19:44:03.094] Checking connection to 10.0.4.2:9999:TCP

[20140711 19:44:03.094] Checking connection to 10.0.4.2:9999:

then it does many: [20140711 19:45:30.605] SyncFilesController[file updated]

[20140711 19:44:33.091] Failed to open tunnel to 10.0.4.2:9999:TCP

[20140711 19:44:33.092] Failed to open tunnel to 10.0.4.2:9999:TCP

[20140711 19:44:43.096] Checking connection to 10.0.4.2:9999:TCP

[20140711 19:44:43.096] Checking connection to 10.0.4.2:9999:TCP

[20140711 19:45:13.093] Failed to open tunnel to 10.0.4.2:9999:TCP

[20140711 19:45:13.093] Failed to open tunnel to 10.0.4.2:9999:TCP

 

What's wrong with this software that it can't communicate across a lan?

Edited by no1youknowz

Share this post


Link to post
Share on other sites

Wow, this platform is seriously garbage. 

 

Does this actually work for anyone?

 

I redid the setup so the IPs are different now.  But really?  I have all these statuses and still no actual real-time syncing here.

[20140712 00:06:33.465] Checking connection to 172.16.1.2:9999:TCP[20140712 00:06:33.465] Checking connection to 172.16.1.2:9999:TCP[20140712 00:06:33.465] Failed to open tunnel to 172.16.1.2:9999:TCP[20140712 00:06:33.467] Failed to open tunnel to 172.16.1.2:9999:TCP[20140712 00:06:43.472] Checking connection to 172.16.1.2:9999:TCP[20140712 00:06:43.472] Checking connection to 172.16.1.2:9999:TCP[20140712 00:06:43.475] Failed to open tunnel to 172.16.1.2:9999:TCP[20140712 00:06:43.475] Failed to open tunnel to 172.16.1.2:9999:TCP[20140712 00:06:43.483] Lost tunnel to 172.16.1.2:9999:uTP[20140712 00:06:43.484] Lost tunnel to 172.16.1.2:9999:uTP[20140712 00:06:44.475] Best tunnel now is 172.16.1.2:32857:TCP

Share this post


Link to post
Share on other sites

Even after doing a re-install on both.

 

Leaving all options as default.

 

I still see:

[20140712 02:16:38.039] ping 10.0.4.82:53785[20140712 02:16:38.044] Incoming connection from 54.80.12.100:53785[20140712 02:16:38.047] SyncFolder::ProcessRequest() received "state_notify" request.[20140712 02:16:38.047] Got state notify from peer 10CD692C43913B30A934047587C7AD1BB4AA10BF - tree:97BD53E8E5C73FA02902DA648750577A7F6276E0 pieces:1E9B93204895DBF60BB7F3C78AB44B94153A1D1A[20140712 02:16:39.047] Send ping to peer (10CD692C43913B30A934047587C7AD1BB4AA10BF) for share D1E85B013F0FD2AC3237C04890620023E8EE552E:[20140712 02:16:39.047] ping 10.0.4.82:53785[20140712 02:16:40.048] Send ping to peer (10CD692C43913B30A934047587C7AD1BB4AA10BF) for share D1E85B013F0FD2AC3237C04890620023E8EE552E:[20140712 02:16:40.048] ping 10.0.4.82:53785[20140712 02:16:41.048] Send ping to peer (10CD692C43913B30A934047587C7AD1BB4AA10BF) for share D1E85B013F0FD2AC3237C04890620023E8EE552E:[20140712 02:16:41.048] ping 10.0.4.82:53785[20140712 02:16:49.566] Failed to open tunnel to 10.0.4.82:53785:TCP[20140712 02:16:55.551] Lost tunnel to 54.80.12.100:46719:TCP[20140712 02:16:55.551] Failed to open tunnel to 67.215.231.242:3000:TCP[20140712 02:16:56.051] Best tunnel now is 54.80.12.100:53785:uTP[20140712 02:16:56.051] Found peer for folder /srv/www/remote/engine 10CD692C43913B30A934047587C7AD1BB4AA10BF 54.80.12.100:53785 direct:1 transport:1[20140712 02:16:56.557] Lost tunnel to 54.80.12.100:53785:uTP[20140712 02:16:57.052] Best tunnel now is 54.80.12.100:53785:uTP[20140712 02:16:57.052] Found peer for folder /srv/www/remote/engine 10CD692C43913B30A934047587C7AD1BB4AA10BF 54.80.12.100:53785 direct:1 transport:1[20140712 02:16:59.052] Checking connection to 10.0.4.82:53785:TCP[20140712 02:17:28.571] Failed to open tunnel to 10.0.4.82:53785:TCP[20140712 02:17:38.075] Checking connection to 10.0.4.82:53785:TCP[20140712 02:18:07.562] Failed to open tunnel to 10.0.4.82:53785:TCP[20140712 02:21:16.135] Requesting peers from tracker 54.225.196.38:3000 for share D1E85B013F0FD2AC3237C04890620023E8EE552E[20140712 02:21:16.135] Using existent uTP tracker connection to 54.225.196.38:3000[20140712 02:21:16.148] Got list of 2 peers for share D1E85B013F0FD2AC3237C04890620023E8EE552E[20140712 02:21:16.148] Peer 1: 54.80.12.100:53785 10CD692C43913B30A934047587C7AD1BB4AA10BF[20140712 02:21:16.148] Peer 1: local IP 10.0.4.82:53785

Lots of failures and even then takes 10 mins to do something.

 

Looks like I'll have to go with either dropbox or aerofs.  This isn't ready for enterprise, not in the least!

Share this post


Link to post
Share on other sites

@no1youknowz

 

I see that you've opened a ticket with our support - please follow support engineer helping you. We'll do our best to resolve your issue.

Share this post


Link to post
Share on other sites

Hi,

 

Thanks for following up.  Even though I am having a hard time trying to get this to work.  I do appreciate someone spending time on this.  

 

Someone contacted me and I did reply back.  So thank you for that.

 

I am trying dropbox, aerofs and btsync, but I am trying to find the best solution for me.  

 

On paper this seems like the best solution for me.

 

1) P2P syncing.

2) Lan syncing.

3) Ability Ignore files/folders.

4) One way syncing.

5) Probably some more things but can't think of it right now.

 

It's just a shame this did not work out of the box for me.

Share this post


Link to post
Share on other sites

So I tried this on redhat 7 on AWS.

Server A:[20140714 22:16:18.981] Fully synced with 100D94CF2BC1090459F46007201BDD17C85C2F29.....[20140714 22:26:18.779] Fully synced with 100D94CF2BC1090459F46007201BDD17C85C2F29Server B: receiving file[20140714 22:16:18.908] Last pieces hash for 10669DBA7714F98C87482CE7EFAC20DE920B7DA0 is 6AC81F034E7D3BE8A550FC4A654A08BBD7B775E6[20140714 22:26:08.323] Merge: processing files message with 1 files[20140714 22:26:08.323] SyncFilesController: Got 1 files from remote (10.0.4.37:48705)[20140714 22:26:08.323] SyncFilesController: Got file from remote (10.0.4.37:48705): hello3.txt state: 1 type: empty total:0 have:0 t:1405391167 mt:1405390702 0000000000000000000000000000000000000000[20140714 22:26:18.706] State sync finished for folder /home/ec2-user/btsync[20140714 22:26:18.706] Last pieces hash for 10669DBA7714F98C87482CE7EFAC20DE920B7DA0 is C63F12B599FCE5FB31858EE6783A56CF5CAA2D40

As you can see.  8 minutes until I file is synced over and these 2 boxes are on the same lan!

Share this post


Link to post
Share on other sites

PLEASE CLOSE.

 

In the end I redesigned the process I require.

 

Your engineer got back to me once and wasn't really much helpful with his initial response.

 

Unless this is dramatically improved in the future, I will not be considering it for my infrastructure.  Which in the future will be around 50 servers and so yes, I will be needing a ROBUST system.

Share this post


Link to post
Share on other sites

@no1youknowz

 

Its a pity we did not manage to help you. Sync team is working hard to make Sync better and more reliable. Hope to see you later with us when Sync becomes good enough to fit your purposes. 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Sign in to follow this