jdyer

Members
 View Profile  See their activity

  • Posts

    19

  • Joined

  • Last visited

 Content Type 

Posts posted by jdyer

  2. Out Of Sync - No Peers Online

    in Sync Troubleshooting

    Posted

    Personally I have about given up in btsync and I am now evaluating Synthing as an alternative.  I can tell you that so far Syncthing ( new name will be pulse ) has simply worked right out of the box, which was a pleasant change compared to my experience with Btsync.  It is very strange, I would think btsync would be far more stable since its the more mature offering but that doesn't seem to be the case... Shame...

  3. Btsync Appears To Be Indexing Ignored Files

    in Sync Troubleshooting

    Posted

    My problem is that every hour there is 35,000 files being created.. So for example at 2014-11-06-2100 I would have 35000 files created 

     

    Example

    /opt/customer/10000/logs/2014-11-06-2100.txt/opt/customer/10001/logs/2014-11-06-2100.txt/opt/customer/10002/logs/2014-11-06-2100.txt

    So this indexing takes a long time and seems to happen regularly and slow down the entire process. I am about to start looking into Syncthing because I am not getting a good feeling about bittorrent.  They support seems unresponsive to these forums and to direct email requests.... 

  4. Indexing After Re-Starting Sync Client

    in Sync Troubleshooting

    Posted

    So I think I am seeing something similar which I am trying to get help w/ in ticket #32426-btsync-appears-to-be-indexing-ignored-files.  My problem is I have a large number of files generated each hour ( logging ) which seems to have to be indexed and takes hours.  the ignoreSync pattern seems to be working by not syncing these files but unfortunately is seems to be indexing them regardless of that config... Its quite disappointing :( 

  6. Out Of Sync - No Peers Online

    in Sync Troubleshooting

    Posted

    I am having the same issue, 1.4.99 with two servers running on AWS.  Initial sync of 150gb between two instances in a VPC took almost 3 days which is absolutely insane 

     

    I can see I am getting 400Mb between the instances

    ## server1Complete![root@wh2 btsync]# iperf -c 10.10.2.145------------------------------------------------------------Client connecting to 10.10.2.145, TCP port 5001TCP window size: 96.4 KByte (default)------------------------------------------------------------[  3] local 10.10.2.210 port 46279 connected with 10.10.2.145 port 5001[ ID] Interval       Transfer     Bandwidth[  3]  0.0-10.0 sec   450 MBytes   376 Mbits/sec[root@wh2 btsync]### server 2[root@ip-10-10-2-145 btsync]# iperf -s------------------------------------------------------------Server listening on TCP port 5001TCP window size: 85.3 KByte (default)------------------------------------------------------------[  4] local 10.10.2.145 port 5001 connected with 10.10.2.210 port 46279[ ID] Interval       Transfer     Bandwidth[  4]  0.0-10.1 sec   450 MBytes   373 Mbits/sec

    here are my configs :

    config1

    {  "device_name": "wh2",  "listening_port" : 44444,  "lan_encrypt_data" : false,  "lan_use_tcp" : true,  "sync_trash_ttl" : 14,  "storage_path" : "/home/btsync",  "pid_file" : "/home/btsync/btsync.pid",  "use_upnp" : false,  "rate_limit_local_peers": false,  "check_for_updates" : false,  "profiler_enabled" : true,  "download_limit" : 0,  "upload_limit" : 0,  "disk_low_priority" : false,  "config_refresh_interval" : 60,  "recv_buf_size" : 24,  "send_buf_size" :24,  "folder_defaults.known_hosts" : "10.10.2.210:44444,10.10.2.145:44444",  "folder_defaults.use_dht" : false,  "folder_defaults.use_lan_broadcast" : false,  "folder_defaults.use_relay" : false,  "folder_defaults.use_tracker" : false,  "webui" : {    "listen" : "0.0.0.0:8888",    "login" : "admin",    "password" : "test"  }}

    config2

    {  "device_name": "webhosting-test",  "listening_port" : 44444,  "lan_encrypt_data" : false,  "lan_use_tcp" : true,  "sync_trash_ttl" : 14,  "storage_path" : "/home/btsync",  "pid_file" : "/home/btsync/btsync.pid",  "use_upnp" : false,  "rate_limit_local_peers": false,  "check_for_updates" : false,  "profiler_enabled" : true,  "download_limit" : 0,  "upload_limit" : 0,  "disk_low_priority" : false,  "config_refresh_interval" : 60,  "recv_buf_size" : 24,  "send_buf_size" :24,  "folder_defaults.known_hosts" : "10.10.2.210:44444,10.10.2.145:44444",  "folder_defaults.use_dht" : false,  "folder_defaults.use_lan_broadcast" : false,  "folder_defaults.use_relay" : false,  "folder_defaults.use_tracker" : false,  "webui" : {    "listen" : "0.0.0.0:8888",    "login" : "admin",    "password" : "tropo"  }}

    Any help would be great... I must say that I had heard great things about btsync but so far my experience has been pretty poor...  I sure hope I am the exception and not the rule :( 

  8. Btsync Appears To Be Indexing Ignored Files

    in Sync Troubleshooting

    Posted

    Still seeing this behavior w/ todays logging, here is an excerpt. 

    [20141105 15:37:15.561] [OnNotifyFileChange] "/opt/customer_home/87162/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.562] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.562] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.562] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.562] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.562] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.562] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.562] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.562] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.562] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.562] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.562] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.562] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.563] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.563] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.563] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.563] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.563] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.563] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.563] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.563] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.563] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.563] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-05-1500.txt", source = "NULL"[20141105 15:37:15.564] SyncFolderNotify: "2014-11-05-1500.txt", event = "IN_MODIFY"[20141105 15:37:15.564] [OnNotifyFileChange] "/opt/customer_home/62700/logs/2014-11-05-1500.txt", source = "NULL"^C

    Does the IgnoreList not truly ignore files ? 

  9. Docs Improvements / Corrections

    in Sync General Discussion

    Posted

    Few things I noticed:

     

    http://sync-help.bittorrent.com/customer/portal/articles/1592522-sync-is-downloading-very-slow

     

     

     

    • Transferring a big number of small files. Download/Upload Speed will be the same for many small files as when transferring a few large files.​

     

    This doesn't make sense in the context of troubleshooting.  This seems to imply that the number or size of files have no impact on sync speed which I don't think is the case.  

     

    ------------------------------------------------

     

     

    http://sync-help.bittorrent.com/customer/portal/articles/1670248-advanced-preferences

     

     

    profiler_enabled     Requires client restart to activate. Starts recording data for speed issue analysis. Data is stored in proffer.dat in storage folder, rotated every 10 minutes

     

    profiler.dat is spelled incorrectly.  

     

    ------------------------------------------------

     

    I will add more to this post as I find them. 

     

    thanks

     

    -John

  13. Btsync Appears To Be Indexing Ignored Files

    in Sync Troubleshooting

    Posted

    So my directory tree has approximately 35000 files at the root ( /opt/customer_home) and each of these folders have a very predictable folder tree at their root [1]. 

     

    I want to ignore a few folders of the root of each account which is why I put logs/* and reports/* in the Ignore list located at the root of the sync ( /opt/customer_home/.sync/IgnoreList ) [2].    
     
    The problem is that the share never seems to finish indexing and transfers are very slow [4].  It has been running for over 36 hours now and is only at 1% with transfer speeds of less then 1mb [3].  My best guess is that because new log files are written every hour to the 35000 folders at the root then btsync is unable to keep up, even despite my configuration telling it specifically to ignore those logs.    I can tell you that btsync is not syncing those logs but it still appears to be indexing them.    I am not sure if this is related to the slow sync I am seeing in my other posting but just incase it wasn't I figured I would create a separate ticket [4]
     
     
     Environment

    • CentOS 6
    • BTSync 1.4.99
    • Servers in Amazon VPC
    • All private traffic opened between hosts
    • Root of share /opt/customer_home
    • Total files to sync 150gb
    • config [5]

     

    References
     
    [1]

    /opt||customer_home||-170129|| |--reports/| |--www/| |--logs|| |--2014-10-31-2200.txt.gz| |--2014-10-31-2300.txt.gz| |--2014-11-01-0000.txt.gz| |--2014-11-01-0100.txt.gz||-170130||--reports/|--www/|--logs||--2014-10-31-2200.txt.gz|--2014-10-31-2300.txt.gz|--2014-11-01-0000.txt.gz|--2014-11-01-0100.txt.gz 

     
    [2]

    [root@wh2 btsync]# cat /opt/customer_home/.sync/IgnoreList/*/logs/*/*/reports/*/*/records/**.tgz*.tar*.cdr.json*.cdr.csv*.txt.tgz*.tar.gz*.pdf*.exe*.dmg*.zip

     
    [3]

    [20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/5000377/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.586] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.586] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.587] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.587] [OnNotifyFileChange] "/opt/customer_home/84575/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.587] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.587] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-04-1400.txt", source = "NULL"[20141104 14:27:43.587] SyncFolderNotify: "2014-11-04-1400.txt", event = "IN_MODIFY"[20141104 14:27:43.587] [OnNotifyFileChange] "/opt/customer_home/5001431/logs/2014-11-04-1400.txt", source = "NULL"

     
    [4]  http://forum.bittorrent.com/topic/31320-extremely-slow-sync/?p=94129

     

    [5]

    {  "device_name": "server1",  "listening_port" : 44444,  "storage_path" : "/home/btsync",  "pid_file" : "/home/btsync/btsync.pid",  "check_for_updates" : false,  "use_upnp" : false,  "download_limit" : 0,  "upload_limit" : 0,  "webui" :  {    "listen" : "0.0.0.0:8888",    "login" : "xxxx",    "password" : "xxxx"  },  "disk_low_priority" : false,  "config_refresh_interval" : 60}

  14. Extremely Slow Sync

    in Sync Troubleshooting

    Posted

    How do you want me to tell you the percentage of BW used? Regarding the disk_priority setting, I have set this to false and it does not appear to have made any difference


    I am not sure if its related but on the "master" server I see the following in the logs 

    [20141103 22:54:54.148] SyncFolderNotify: "2014-11-03-2200.txt", event = "IN_MODIFY"[20141103 22:54:54.148] [OnNotifyFileChange] "/opt/customer_home/57801/logs/2014-11-03-2200.txt", source = "NULL"[20141103 22:54:54.148] SyncFolderNotify: "2014-11-03-2200.txt", event = "IN_MODIFY"[20141103 22:54:54.148] [OnNotifyFileChange] "/opt/customer_home/57801/logs/2014-11-03-2200.txt", source = "NULL"[20141103 22:54:54.148] SyncFolderNotify: "2014-11-03-2200.txt", event = "IN_MODIFY"[20141103 22:54:54.148] [OnNotifyFileChange] "/opt/customer_home/57801/logs/2014-11-03-2200.txt", source = "NULL"[20141103 22:54:54.148] SyncFolderNotify: "2014-11-03-2200.txt", event = "IN_MODIFY"[20141103 22:54:54.148] [OnNotifyFileChange] "/opt/customer_home/57801/logs/2014-11-03-2200.txt", source = "NULL"[20141103 22:54:54.148] SyncFolderNotify: "2014-11-03-2200.txt", event = "IN_MODIFY"[20141103 22:54:54.148] [OnNotifyFileChange] "/opt/customer_home/57801/logs/2014-11-03-2200.txt", source = "NULL"

    Please note that /opt/customer_home is the root of the share and my IgnoreFile is as follows:

    [root@wh2 btsync]# cat /opt/customer_home/.sync/IgnoreList/*/logs/*/*/reports/*/*/records/**.tgz*.tar*.cdr.json*.cdr.csv*.txt.tgz*.tar.gz*.pdf*.exe*.dmg*.zip

  15. Blank Gui After Update To 1.4

    in Sync Troubleshooting

    Posted

    I still get this, and it's very annoying... 

     

    OSX 10.10

    Chrome 38

    Firefox 34

     

    I should mention there is zero helpful logging anywhere.  This was after I spend 20 minutes looking for the  seemingly undocumented configuration to even get the linux client into debug mode...   All and all the user experience so far isnt that great IMHO 

  17. [Solved] Syncignore Not Working As Expected

    in Sync Troubleshooting

    Posted

    so I have a folder structure as follows

    /opt    |    |customer_home|                  |-170129|                          |--reports/                          |--www/                          |--logs/

    however under customer home there are about 50,000 account id's.  My intent is to ignore everything under logs and reports but sync /www

     

    I have placed my .SyncIgnore file at the root of my sync ( /opt/customer_home/.SyncIgnore ) with the following

    /*/logs/*/reports*.tgz*.tar*.cdr.json*.cdr.csv*.txt.tgz*.tar.gz*.pdf*.exe*.dmg*.zip

    but it is still syncing files under logs and reports.  I am using client 1.4 and syncing between two CentOS 6 systems.   Any idea what I am doing wrong?

  19. Sync Between Hosts On Lan Securely

    in Sync General Discussion

    Posted

    Question guys, 

     

    I have the following config.json for my two nodes.   I am wondering if I am still sending any data over the internet or would all my traffic stay within my local lan ? From what I understand using the known_hosts directive, along with search_lan, I am not going to be using any public trackers.  Is this correct? 

     

    Thanks!

     

    -John

    {    "device_name": "vpn1.us-west-2xxxxxxxx",    "listening_port": 55555,    "storage_path": "/etc/btsync",    "pid_file": "/var/run/btsync/btsync.pid",    "check_for_updates": false,    "use_upnp": false,    "download_limit": 0,    "upload_limit": 0,    "disk_low_priority": false,    "folder_rescan_interval": 120,    "lan_encrypt_data": true,    "lan_use_tcp": true,    "rate_limit_local_peers": false,    "sync_max_time_diff": 300,    "sync_trash_ttl": 30    ,"shared_folders":    [    {        "secret" : "xxxxx",        "dir" : "/etc/somewhere/",        "use_relay_server" : false,        "use_tracker" : false,        "use_dht" : false,        "search_lan" : true,        "use_sync_trash" : false        ,        "known_hosts" : ["10.1.0.10:55555", "10.10.0.10:55555"]    }