[Solved] How do the machines find each other via the "Secret" code?


Recommended Posts

So that's the question, How do the machines find each other simply via the secret? What is that magic that makes these wheels go round?

If there is no central server that holds the hashes pulled from the secrets and then matches the hashes if it finds another one like it, i have no clue how it works.

These hashes have to be somewhere and that somewhere, i believe, has to have a pretty powerful "database" of sorts, the world is gigantic, a lot of hashes, a lot of shares etc.

Whatever way, there are too many hashes to go through via one client, it seems impossible without a central server.

Anyone in here to be able to explain this to me?

Link to comment
Share on other sites

there's a central server to discover other computers that share the secret.

but you can use it on a LAN aswell and btsync will only try to find computers on the LAN. There are differents settings to tell what you want btsync to do.

Link to comment
Share on other sites

Submit data to a tracker with a hashed version of the secret, the server then gives the client all the IPs that have sent that same code in recently.

...but this only applies IF you're using the tracker server option - BitTorrent Sync will also work (i.e. locally within your own LAN) without negotiating via a central tracker server ...so individual BitTorrent Sync clients are capable of processing "secrets" themselves

Link to comment
Share on other sites

Peer discovery:

BTSync uses several methods of peer discovery.

  • Known hosts: this is the simplest; you enter a Port number and an IPaddress or DNS hostname and BTSync attempts to contact this host (This should also work with dynamic DNS if you wish)
  • Search lan: this sends out multicast packets to the local lan (and rarely some connected ones) on port 3838. If a client receives once of these it attempts a normal connection to that peer.
  • Tracker: BTSync sends the info hash of the share (basically the hash of the secret) to t.usyncapp.com. That host keeps a list of the IP:port pairs that have contacted it with that hash and gives them out to everyone interested.
  • DHT: (Distributed Hash Table) this is very similar in concept to a tracker, except the hashs are not stored on one central server but distributed across all the peers in all the swarms. There does need to be a starter peer (one of which I expect is hosted by bittorrent.com) but once started the network is self supporting. (This one is real magic :) )

use_lan_tcp is NOT use for peer discovery; it is a way of speeding up connections to peers that have already been contacted with UDP (like turning off lan_use_encryption).

The relay server is NOT used for peer discovery; it's for working around stupidly obstructive NAT and firewall devices.

Link to comment
Share on other sites

To: rdebath,

Tracker: BTSync sends the info hash of the share (basically the hash of the secret) to t.usyncapp.com. That host keeps a list of the IP:port pairs that have contacted it with that hash and gives them out to everyone interested.

So basically when that central server is down, nothing works anymore? Taking in account that there are no LAN shares and DHT is disabled etc? Also, isn't there too much processing done on the server side and shouldn't the server get overloaded easily?

I have tried to get "Use predefined hosts" to work under windows, but i can't get it to work. As far as i understand, every other option can be disabled and only the predefined host has to have the host name and the port and everything should work. Which i have and i have enabled the same port in the router, firewall and made the client listen on that port on the other machine and no syncing seems to be happening.

Anyway, about the main topic, so basically there still is a central server and this application isn't that much of a miracle?

Link to comment
Share on other sites

Peer discovery:

BTSync uses several methods of peer discovery.

  • Known hosts: this is the simplest; you enter a Port number and an IPaddress or DNS hostname and BTSync attempts to contact this host (This should also work with dynamic DNS if you wish)
  • Search lan: this sends out multicast packets to the local lan (and rarely some connected ones) on port 3838. If a client receives once of these it attempts a normal connection to that peer.
  • Tracker: BTSync sends the info hash of the share (basically the hash of the secret) to t.usyncapp.com. That host keeps a list of the IP:port pairs that have contacted it with that hash and gives them out to everyone interested.
  • DHT: (Distributed Hash Table) this is very similar in concept to a tracker, except the hashs are not stored on one central server but distributed across all the peers in all the swarms. There does need to be a starter peer (one of which I expect is hosted by bittorrent.com) but once started the network is self supporting. (This one is real magic :) )

use_lan_tcp is NOT use for peer discovery; it is a way of speeding up connections to peers that have already been contacted with UDP (like turning off lan_use_encryption).

The relay server is NOT used for peer discovery; it's for working around stupidly obstructive NAT and firewall devices.

thanks! this is very well written. i think an explanation like this should be included into a BTsync manual.

Link to comment
Share on other sites

The BTSync tracker is the same as a Bittorrent tracker, it does almost no work.

For each combination of peer and share it stores:

  1. The hash for the share
  2. The IP address of the peer
  3. The port number of the peer
  4. The time to delete this record.

Once two peers have been introduced to each other the tracker has nothing more to do. If the tracker goes down after the peers have linked up there's no problem. That DNS name resolves to three independent servers which will (probably) be sharing information. But, I expect, (deal killer if not!) there will be a configuration option to allow other trackers later on.

As I said the real magic is the DHT mode where if you know or remember the location of any node in the network you can use it to find the rest of the network.

I've tested the known hosts connection with Windows-7, it's working fine for me. I've tested it with only one side knowing the location of the other and that works too (depending on the firewall). Remember you have to open UDP ports not TCP ports and usually tell the BOTH clients the public IP addresses of the other so they can work together to poke holes through NAT or connection tracking. If the public IPs are dynamic you can use services like dyndns.org to get a constant dns name.

If it's still not working you might need to raise a bug report.

Link to comment
Share on other sites

Ok, just in case, i'll describe my situation:

I have PC A and PC B.

I start by shutting down the client on PC B and create the shared folder on PC A, then untick ALL the options in "General" tab and leave only the "Predefined hosts" ticked and i fill in the correct host info and port to PC B. I then shut down PC A BTSync client.

I now proceed to PC B and i'll add the new shared folder with the secret key i got from PC A. When the folder is ready, i untick ALL the options under "General" tab, as they are not needed there (correct me if i'm wrong, but i don't think PC B needs to connect to itself and all the other options are not needed as well, since i wwant to use predefined hosts). I save the settings and double-check if i have my PC B client listening on the correct port, yes it is. I now shut down the PC B BTSync client and start it up again.

I now go to my PC A where i start up BTSync client and add some random mp3 files into the shared folder. PC A BTSync client successfully recognizes and indexes the files. I am now looking at PC B to see if these files sync to that and they DO NOT.

Just in case i have both TCP AND UDP ports open in my router. I have double-checked to see if my router has UPnP enabled and as well as BTSync client. Port forwarding can't be the issue here, as i have other services running and i am accessible to the WAN.

Which is weird though, is that when using a service like CanYouSeeMe(dot)org which checks for open ports, it says the port is closed on my BTSync client, should it say that?

So can anyone else use the EXACT step-by-step guide to test if the predefined host method is working for them or not.

Also to make sure NO OTHER shared folders are there in the "Shared Folders" list. It is also CRUCIALLY important to SHUT DOWN the other BTSync client, as if you test this on LAN or anywhere else and have the default options ticked, the clients exchange information and find themselves, so when you test with predefined hosts without shutting the clients down, they will see each other and the method "seems to be working" even though it is not (for me at least).

Link to comment
Share on other sites

Okay, just for you I'll do it again and write down what I'm doing ...

Two new PCs never had it installed before.
Add rules on both firewall to direct a UDP port from outside to the PCs.
Download vsn 1.0.132 to both PCs.
Install to both PCs with different random keys, they don't WANT to talk to each other.
Turn off UPnP; I don't need it.
Configure the port numbers on both PCs.
Turn off ALL ticks on the General tab for the share on both PCs. -- save
Put a file in the share of PC No.1.
Set the secrets on both PCs to LOVGDZGJXE7T6BUCEUFWZ6BTDN7OND6Y == base32(SHA1("password")) -- save
Leave them for a while.
...
...
PCs are NOT talking to each other.
...
...
Still not talking.
...
Add DNS hostname and port for the firewall of PC1 as a known host on PC2.
Moments later the PCs see each other and up pops the message about a file transfer (it was a very small file).

Add a couple of other files and did some movements; no problems.

Then I removed the known host turned them off, got them not talking. Then did it the other way round.

Notice in both cases I had the peers running but not talking then talking immediately I entered the host name.

Plus I was running tcpdump on both firewalls.

The website CanYouSeeMe(dot)org only checks for TCP ports not UDP

I haven't found one that does UDP.

But as you've forwarded both and BTSync opens both CanYouSeeMe will see it if you've done it right.


But I'm going to leave mine running to see how well it follows when my IP changes.

Link to comment
Share on other sites

rdebath I got this working and i'll explain how and why i am sorry for the inconvenience i caused.

To everyone else as well, my problem was still my router. It turns out (since it's a new one and i'm not that familliar with it) it needs a restart after applying open ports to the settings list (why in the hell, i'll never know).

So, as a last step in troubleshooting, i decided to restart my router, all the connections went down, lost my TV's for a few minutes, a lot of hassle came along with it (as i have many webservices running with custom configuration etc), but it works at least, the files synced immediately.

I now feel like the person who has gone through everything troubleshooting his non-working PC and then figures out he has to turn the damn thing on.

Anyway thanks to everybody and special thanks to rdebath!

Hopefully eventually we can set up our own BTSync servers (the ones that store the hashes etc) where the client connects and also to customise a portable installation with custom conf file with custom everything.

Link to comment
Share on other sites

  • 2 weeks later...

I keep getting pointed to this link for the fix to my issue but i am not fully sure what we need to do to fix the issue.

i do not fully understand how this could be a firewall issue if computers inside the same domain controller can see each other.

This is the issue:

we have 6 domain controller servers under one domain

computers connected the domain controller site sync with each other no issue

But between sites the sync software does not see each other.

why?

cant be a firewall issue because the computer talk to each other at that site without any adjustment of our domain firewall.

the only issue is if PC from Site A does not see PC from Site B in the BitTorrent Sync software.

why?

how do i get these computers to talk to each other?

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.