jdyer

Members
  • Posts

    19
  • Joined

  • Last visited

Everything posted by jdyer

  1. Based on my experience the last 4 days I can say with certainty that version 1.4 is a total disaster. I am honestly not sure how a build without the ability to even perform its central feature ( sync ) made it out of their QA department...
  2. Personally I have about given up in btsync and I am now evaluating Synthing as an alternative. I can tell you that so far Syncthing ( new name will be pulse ) has simply worked right out of the box, which was a pleasant change compared to my experience with Btsync. It is very strange, I would think btsync would be far more stable since its the more mature offering but that doesn't seem to be the case... Shame...
  3. My problem is that every hour there is 35,000 files being created.. So for example at 2014-11-06-2100 I would have 35000 files created Example /opt/customer/10000/logs/2014-11-06-2100.txt/opt/customer/10001/logs/2014-11-06-2100.txt/opt/customer/10002/logs/2014-11-06-2100.txtSo this indexing takes a long time and seems to happen regularly and slow down the entire process. I am about to start looking into Syncthing because I am not getting a good feeling about bittorrent. They support seems unresponsive to these forums and to direct email requests....
  4. So I think I am seeing something similar which I am trying to get help w/ in ticket #32426-btsync-appears-to-be-indexing-ignored-files. My problem is I have a large number of files generated each hour ( logging ) which seems to have to be indexed and takes hours. the ignoreSync pattern seems to be working by not syncing these files but unfortunately is seems to be indexing them regardless of that config... Its quite disappointing
  5. I am having the same issue, 1.4.99 with two servers running on AWS. Initial sync of 150gb between two instances in a VPC took almost 3 days which is absolutely insane I can see I am getting 400Mb between the instances ## server1Complete![root@wh2 btsync]# iperf -c 10.10.2.145------------------------------------------------------------Client connecting to 10.10.2.145, TCP port 5001TCP window size: 96.4 KByte (default)------------------------------------------------------------[ 3] local 10.10.2.210 port 46279 connected with 10.10.2.145 port 5001[ ID] Interval Transfer Bandwidth[ 3] 0.0-10.0 sec 450 MBytes 376 Mbits/sec[root@wh2 btsync]### server 2[root@ip-10-10-2-145 btsync]# iperf -s------------------------------------------------------------Server listening on TCP port 5001TCP window size: 85.3 KByte (default)------------------------------------------------------------[ 4] local 10.10.2.145 port 5001 connected with 10.10.2.210 port 46279[ ID] Interval Transfer Bandwidth[ 4] 0.0-10.1 sec 450 MBytes 373 Mbits/sechere are my configs : config1 { "device_name": "wh2", "listening_port" : 44444, "lan_encrypt_data" : false, "lan_use_tcp" : true, "sync_trash_ttl" : 14, "storage_path" : "/home/btsync", "pid_file" : "/home/btsync/btsync.pid", "use_upnp" : false, "rate_limit_local_peers": false, "check_for_updates" : false, "profiler_enabled" : true, "download_limit" : 0, "upload_limit" : 0, "disk_low_priority" : false, "config_refresh_interval" : 60, "recv_buf_size" : 24, "send_buf_size" :24, "folder_defaults.known_hosts" : "10.10.2.210:44444,10.10.2.145:44444", "folder_defaults.use_dht" : false, "folder_defaults.use_lan_broadcast" : false, "folder_defaults.use_relay" : false, "folder_defaults.use_tracker" : false, "webui" : { "listen" : "0.0.0.0:8888", "login" : "admin", "password" : "test" }}config2 { "device_name": "webhosting-test", "listening_port" : 44444, "lan_encrypt_data" : false, "lan_use_tcp" : true, "sync_trash_ttl" : 14, "storage_path" : "/home/btsync", "pid_file" : "/home/btsync/btsync.pid", "use_upnp" : false, "rate_limit_local_peers": false, "check_for_updates" : false, "profiler_enabled" : true, "download_limit" : 0, "upload_limit" : 0, "disk_low_priority" : false, "config_refresh_interval" : 60, "recv_buf_size" : 24, "send_buf_size" :24, "folder_defaults.known_hosts" : "10.10.2.210:44444,10.10.2.145:44444", "folder_defaults.use_dht" : false, "folder_defaults.use_lan_broadcast" : false, "folder_defaults.use_relay" : false, "folder_defaults.use_tracker" : false, "webui" : { "listen" : "0.0.0.0:8888", "login" : "admin", "password" : "tropo" }}Any help would be great... I must say that I had heard great things about btsync but so far my experience has been pretty poor... I sure hope I am the exception and not the rule
  6. Any ideas what I should try next?
  7. Still seeing this behavior w/ todays logging, here is an excerpt. [20141105 15:37:15.561] [OnNotifyFileChange] "/opt/customer_home/87162/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.562] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.562] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.562] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.562] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.562] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.562] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.562] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.562] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.562] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.562] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.562] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.562] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.563] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.563] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.563] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.563] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.563] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.563] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.563] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.563] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.563] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.563] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.564] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.564] [OnNotifyFileChange] "/opt/customer_home/62700/logs/2014-11-05-1500.txt", source = "NULL"^CDoes the IgnoreList not truly ignore files ?
  8. Few things I noticed: http://sync-help.bittorrent.com/customer/portal/articles/1592522-sync-is-downloading-very-slow This doesn't make sense in the context of troubleshooting. This seems to imply that the number or size of files have no impact on sync speed which I don't think is the case. ------------------------------------------------ http://sync-help.bittorrent.com/customer/portal/articles/1670248-advanced-preferences profiler.dat is spelled incorrectly. ------------------------------------------------ I will add more to this post as I find them. thanks -John
  9. Would it be possible to add something in the logs that say this in a innocuous message and can be disregarded ?
  10. So my directory tree has approximately 35000 files at the root ( /opt/customer_home) and each of these folders have a very predictable folder tree at their root [1]. I want to ignore a few folders of the root of each account which is why I put logs/* and reports/* in the Ignore list located at the root of the sync ( /opt/customer_home/.sync/IgnoreList ) [2]. The problem is that the share never seems to finish indexing and transfers are very slow [4]. It has been running for over 36 hours now and is only at 1% with transfer speeds of less then 1mb [3]. My best guess is that because new log files are written every hour to the 35000 folders at the root then btsync is unable to keep up, even despite my configuration telling it specifically to ignore those logs. I can tell you that btsync is not syncing those logs but it still appears to be indexing them. I am not sure if this is related to the slow sync I am seeing in my other posting but just incase it wasn't I figured I would create a separate ticket [4] Environment CentOS 6BTSync 1.4.99Servers in Amazon VPCAll private traffic opened between hostsRoot of share /opt/customer_homeTotal files to sync 150gbconfig [5] References [1] /opt||customer_home||-170129|| |--reports/| |--www/| |--logs|| |--2014-10-31-2200.txt.gz| |--2014-10-31-2300.txt.gz| |--2014-11-01-0000.txt.gz| |--2014-11-01-0100.txt.gz||-170130||--reports/|--www/|--logs||--2014-10-31-2200.txt.gz|--2014-10-31-2300.txt.gz|--2014-11-01-0000.txt.gz|--2014-11-01-0100.txt.gz [2] [root@wh2 btsync]# cat /opt/customer_home/.sync/IgnoreList/*/logs/*/*/reports/*/*/records/**.tgz*.tar*.cdr.json*.cdr.csv*.txt.tgz*.tar.gz*.pdf*.exe*.dmg*.zip [3] [20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/5000377/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.587] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.587] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.587] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.587] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.587] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.587] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-04-1400.txt", source = "NULL" [4] http://forum.bittorrent.com/topic/31320-extremely-slow-sync/?p=94129 [5] { "device_name": "server1", "listening_port" : 44444, "storage_path" : "/home/btsync", "pid_file" : "/home/btsync/btsync.pid", "check_for_updates" : false, "use_upnp" : false, "download_limit" : 0, "upload_limit" : 0, "webui" : { "listen" : "0.0.0.0:8888", "login" : "xxxx", "password" : "xxxx" }, "disk_low_priority" : false, "config_refresh_interval" : 60}
  11. How do you want me to tell you the percentage of BW used? Regarding the disk_priority setting, I have set this to false and it does not appear to have made any difference I am not sure if its related but on the "master" server I see the following in the logs [20141103 22:54:54.148] SyncFolderNotify: "2014-11-03-2200.txt", event = "IN_MODIFY"[20141103 22:54:54.148] [OnNotifyFileChange] "/opt/customer_home/57801/logs/2014-11-03-2200.txt", source = "NULL"[20141103 22:54:54.148] SyncFolderNotify: "2014-11-03-2200.txt", event = "IN_MODIFY"[20141103 22:54:54.148] [OnNotifyFileChange] "/opt/customer_home/57801/logs/2014-11-03-2200.txt", source = "NULL"[20141103 22:54:54.148] SyncFolderNotify: "2014-11-03-2200.txt", event = "IN_MODIFY"[20141103 22:54:54.148] [OnNotifyFileChange] "/opt/customer_home/57801/logs/2014-11-03-2200.txt", source = "NULL"[20141103 22:54:54.148] SyncFolderNotify: "2014-11-03-2200.txt", event = "IN_MODIFY"[20141103 22:54:54.148] [OnNotifyFileChange] "/opt/customer_home/57801/logs/2014-11-03-2200.txt", source = "NULL"[20141103 22:54:54.148] SyncFolderNotify: "2014-11-03-2200.txt", event = "IN_MODIFY"[20141103 22:54:54.148] [OnNotifyFileChange] "/opt/customer_home/57801/logs/2014-11-03-2200.txt", source = "NULL"Please note that /opt/customer_home is the root of the share and my IgnoreFile is as follows: [root@wh2 btsync]# cat /opt/customer_home/.sync/IgnoreList/*/logs/*/*/reports/*/*/records/**.tgz*.tar*.cdr.json*.cdr.csv*.txt.tgz*.tar.gz*.pdf*.exe*.dmg*.zip
  12. I still get this, and it's very annoying... OSX 10.10 Chrome 38 Firefox 34 I should mention there is zero helpful logging anywhere. This was after I spend 20 minutes looking for the seemingly undocumented configuration to even get the linux client into debug mode... All and all the user experience so far isnt that great IMHO
  13. Same issue here syncing 150gb worth of files between two AWS servers in the same geographic region
  14. so I have a folder structure as follows /opt | |customer_home| |-170129| |--reports/ |--www/ |--logs/however under customer home there are about 50,000 account id's. My intent is to ignore everything under logs and reports but sync /www I have placed my .SyncIgnore file at the root of my sync ( /opt/customer_home/.SyncIgnore ) with the following /*/logs/*/reports*.tgz*.tar*.cdr.json*.cdr.csv*.txt.tgz*.tar.gz*.pdf*.exe*.dmg*.zipbut it is still syncing files under logs and reports. I am using client 1.4 and syncing between two CentOS 6 systems. Any idea what I am doing wrong?
  15. I am not sure I understand then, if I have tracker set to false do I have any dependency on being able to connect to a public tracker ? I am trying to guarantee that traffic never leaves my private network for this
  16. Question guys, I have the following config.json for my two nodes. I am wondering if I am still sending any data over the internet or would all my traffic stay within my local lan ? From what I understand using the known_hosts directive, along with search_lan, I am not going to be using any public trackers. Is this correct? Thanks! -John { "device_name": "vpn1.us-west-2xxxxxxxx", "listening_port": 55555, "storage_path": "/etc/btsync", "pid_file": "/var/run/btsync/btsync.pid", "check_for_updates": false, "use_upnp": false, "download_limit": 0, "upload_limit": 0, "disk_low_priority": false, "folder_rescan_interval": 120, "lan_encrypt_data": true, "lan_use_tcp": true, "rate_limit_local_peers": false, "sync_max_time_diff": 300, "sync_trash_ttl": 30 ,"shared_folders": [ { "secret" : "xxxxx", "dir" : "/etc/somewhere/", "use_relay_server" : false, "use_tracker" : false, "use_dht" : false, "search_lan" : true, "use_sync_trash" : false , "known_hosts" : ["10.1.0.10:55555", "10.10.0.10:55555"] }