chillicane

Members
  • Posts

    18
  • Joined

  • Last visited

Everything posted by chillicane

  1. thanks, ill collate that info. I will say however that im not using tracker or relay servers on the shares im only relying on peer to peer peer exchange...like thats not a mouthful!
  2. I just checked my test machine and ive noticed it has not increased the number of entries in the devices tab - i would have hoped after 4 days it would have done some kind of peer exchange but it appears it has not occured! Tearing my hair out on this one!
  3. Symptoms started as a massive increase of the sync time and bandwidth used on the 'server' which intially changes the files (all other peers are read only). Also a seperate monitoring package showed some peers were no longer getting the latest version of the file and upon further investigation their 'devices' tab was empty and stayed that way over time (days). Im sure i read somewhere that the current linux version has a bug where peers are not 'expirng' from the devices list over time as they normally would so i have another theory now. Given the 50 peer limit, i think ive accidently created a set of 50 peers which will all sync fine but from 51 onwards they get blocked by the 'server' due to peer limits as shown in debug log (assigned as server by 'use predefiined hosts' option on all 'clients'). To work around this in the mean time i have given peers a few other random peers in 'use predefined hosts' section in the hope they will eventually get a link back to a peer which has access to the 'server' and amongst themselves go back to function as they did before. This is not ideal but it has the added side effect of making the overall bitstorm more robust in the long term. Implementing that fix is ongoing and i wont really know if its worked for a few days when i check to see if more peers have been exchanged on my sample peers. I know devs dont want to raise the peer limit as they are working on a (potentiall monitized) 'corporate' version of btsync which is great and certainly worth considering if they have the necesary features and the price is right but we dont actually have that version yet and ive been using this for over a year to deliver static files to clients in the best way i can imagine. I know it pisses off every cold calling/spamming corporate pricks who keep trying to sell me cloud based sync solutions at per year volume licensing! Right now i would love to see a private tracking only component which i could use to co-ordinate peers (i know theres the third party one). Overall im extremely happy with this program, its a fantiasticly useful application/adaption of the torrent protocol.
  4. after watching for a few days it looks like the peer list entires are expiring but not being refreshed when peers should be exchanging lists with each other. If i add a peer manually it picks it up straight away. Could this be a hangup because of upgrading? Is there a way i can force refresh the peer lists, delete some files perhaps?
  5. Alright, by doing some DNAT trickery i got the peers talking straight to a windows source and the PEX seems to have worked better - based on a few samples i checked. Prhaps the issue is only in the linux sync?
  6. I have ~160 peers (windows) which all get their files from a central server (linux) which in turn gets its files from a handful of internal machines (windows) with the view that the peers will share the load (bandwidth) amongst themselves. I just upgraded them from 1.2.68 to 1.3.106 and on the previous version it worked pretty well, barring a couple of nodes which must have some environmental issues. Now im seeing nodes dropping all their peers from their device lists and only syncing with the central server which isnt exactly in the spirit of what we're trying to achieve here! I only upgraded because i read some release notes somewhere about reduced 'chatter' amongst peers but its not looking good so far. Happy to provide more information to assist in this one! Update - manually adding peers to the predfined hosts works for that peer but it does not kick start the peer exchange process. Update again - am bypassing the linux sync server to see if it will work better with only the windows version, will update
  7. I have witnessed this on a windows installation (a few versions ago however) but i cannot offer any explanation. Certainly not normal behaivour tho
  8. I would also like some clarification on this issue - it reads as Modifications made in such a folder will not be displayed on other devices in sync This suggests as though perhaps deleting files (an act of modification on the read only side) will not cause a refresh of the data because only data changed on the 'server' side will signify a change.
  9. I am positive that peers exchange peer lists with each other or at the very least, the server listed as 'use prededifend hosts' shares its peer list. Its working for me again now ive flushed the peer lists from sync.dat All my read only endpoints are only set to use lan sync and predifined host - no dht, no tracker, no relay - and it definetly works. Most endpoints arent even on the same lan as others, they go out thru the internet and i use port forwarding to allow a bidirection connection. I think the read only thing is simply a way of saying, if we change files in here dont push them out - just ignore, they should still seed for other peers. I have definetly watched them download off other read only peers when syncing. My problem appears to have simply been that the cached peer list was dirty after the upgrade and there was no clean way to flush it without removing the folder shares.
  10. This stuff gets really tricky to discuss without a diagram but here goes. My read only nodes were only supposed to ever recieve data - not send it, check. Other nodes were definetly able to replicate thru nodes which only had a read only key. What might be different in my case was that all nodes have access to my 'main server' which theoritically should have a complete list of all peers which it will share with all other peers so they can, um...., find peers! Each read only share had an entry in 'use predefined hosts' which points back to my 'main server' so it should always be contactable. Also once an initial peer listing has been garnered even if there is a break in communications somewhere, each peer should be able to reconstruct a compelte list from all the other peers. But that initial sync of peer listsings comes from the 'main server' Make sense? Are we even talking about the same thing? Boy i hope so coz its starting to give me a headache!
  11. Yes i understand that - im talking about the 30 odd endpoints ive already upgraded are not talking to each other. In a genius move i tried deleting some of the settings files on my install here and lost my keys, which of course i had neglected to back up anywhere so it looks like i'll be starting again anyway. Rookie mistake and boy do i feel stupid. Anyway, after re-adding a few i checked and they appear to be syncing better now so im guessing its the data in sync.dat isnt being refreshed properly when i run the new binary. Moot point now coz ill have to re-add them all. It would be nice do have 1 file that has peer caching seperate to the file that has shared folder data so we can flush the peer data without removing the shared folder data.
  12. Im in the process of upgrading ~165 btsync installs from 1.1.15 to 1.1.48. All of the 'client' endpoints have readonly keys to sync the data from the 'server' side - the idea being they can share the actual load of transferring bits amongts themselves. Unfortunatley, so far at least, the 'client' endpoints are only connecting to the 'server' device and not each other - they are not showing each other in their respective devices tab. This was working prior to the upgrade and ive made no configuration changes during the process. Any ideas?
  13. This ones just a theory but i noticed in the logs that it appears as tho the internal 'peer list' is getting full and its ignoring new peers once it hits a certain count. Perhaps as this occurs older peers are dropping off the list and newer ones being appended and this processes keeps repeating so the same rather large list of peers is just being cycled over and over again. Clutching at straws now but hopefully i can find an answer!
  14. I should clarify these are all windows machines. I just fired up the ol' wireshark on my pc and straight away 1500 packets in 10 seconds of 'meta data' for 'BSYNC' for all different endpoints - and no files are being synced currently. If i was syncing more than 50mb a day the excess chatter wouldnt be an issue but at the moment it looks like more bandwidth is spent in syncing peer lists and other meta data than actual file data
  15. Update! I checked the debug log approx 28 hours after turning it on and wow, the log file is over 650mb! Somethings obviously not right here but looking at the contents of the log file, all seems to be normal stuff. Lots of Sending broadcast ping for share and Max peers reached, skipping peer and ping 1xx.1xx.1xx.1xx:145xx directly So it appears to me the protocol is just extrememly chatty. Problem is its using much more bandwidth in status updating than it is in transferring actual data! Am i missing something here?
  16. More Infos! I checked MY btsync (which is the source of all truth in this instance) and found 6 other peers which were 'stuck' on 17.8mb upload in the devices tab. On all of the peer machines the last sync was 9 days ago so something was definetly wrong. I tried many things to force them to resync (on the peer machine, not mine), deleted the cache, restarting all the things etc but in the end i deleted the shared folder, recreated it and its now syncing. This still may not be the cause of my data usage problem but there is definetly something weird going on where some peers will get stuck and simply refuse to continue until the shared folder is recreated. Perhaps something lost connection here at my side 9 days ago which has confused peers?
  17. Im currently using btsync to replicate an 18mb file which is updated daily amongst ~23 locations. Only 1 machine is the source of truth and all other peers are using the read-only key. First up it works brilliantly, a task which would previously take at least half a day (with many sites simply failing to update for various technical glitches) now takes around 30 minutes and we can be 100% sure that every site has recieved the file correctly. However im noticing a trend at some sites which are reporting excessive up/down traffic consistently across the day where before that traffic was not present. Taking an example site, looking at my ISPs traffic report for the previous 10 days i see the following Download: Averaging 24.72mb per hour. Upload: Averaging 25.10mb per hour. Whilst the byteage increases around the time i know the file should be syncing, im still seeing consistent data usage at other times, say 1.6mb every 5 minute block (which is how the ISP report is broken up), even throughout the night which rules out incidental usage from another program. I have a few theories: 1. The metadata syncing is actually using too much data, too often. 2. On my example site in the 'Devices' tab - 'Status' column, one peer is stuck on 'U: 17.8mb' and it now appears as though that peer is having trouble keeping its internet connection up. So im wondering if my example site is continuously trying to send file pieces thru UDP to that dodgy site but also constantly failing due to internet dropouts, rinse, repeat! If either of my theories are true perhaps we can have.. 1. An option to specifiy how often to check our peers for new information, peer lists, updated files etc -perhaps i dont need to do it every 10 minutes, an hour may be sufficient. 2. If a peer is constantly requesting the same file peice over a certain amount of time, we assume the peer is broken and either silently stop trying to interact/and or tell the user. Ive enable debug logging on my example site so will have more data soon if requested. Any insights etc will be greatly apprecated, Thanks, Paul.