A thought it was secure ?

[rdebath]

Having said all that, he may actually have raised a point.

It looks to me that this secret is being used in a manner that makes it susceptible to birthday attacks (ie there isn't a "user name" ) a birthday attack halves the number of bits in the probability of a collision so this may be susceptible to an attack of the order of 2⁶⁴.

While this is likely to still be enough, some cryptologists would disagree. This is the reason that SHA uses 160 or even 256 bits for it's hash generation.

[GreatMarko]

@rdebath - I really don't have a clue what you're going on about in your last two posts!!??

But if you read the previous comments in this thread from both mysel and Kos, you will see that the next release of SyncApp will not enforce a specific Secret length (as it does at present), and will also permit symbols as well as alpha-numeric characters.

As I outlined previously, a 6 character long secret would allow for around 56,800,235,584 unique "secrets" (nearly 57 BILLION!) (and that only assumes a secret made up of just A-Z, a-z, and 0-9 characters. Add symbols into that mix and increase the secret length further, and the possible unique combinations are just too big to realisticlly express here!

So, given the upcoming flexibility of secret lengths, how secure a user's SyncApp is will to some extent be up to the user! Longer secrets will make it more secure than shorter secrets... so why not just create and use secrets with a length of say 2048 characters(!)

If users and/or cryptologists "would disagree" as to how likely it would be for someone (or a computer) to "guess" someone else's secret given these forthcoming improvements to secrets - if they're really THAT worried about it, they can always choose not to use SyncApp!

[PeterVerhees]

Then again,

When the length of the key is way longer than the amount of bits used by the form of encryption, you gain nothing.

Example:

When we use a 256 bits encryption without any form of block cipher-based mode of chaining (which is VERY unlikely, but for the purpose of the example necessary because it makes attacking the cypher much more difficult) you could use a 4096 bits key to encrypt. However, this results only in a 256 bits encryption with no further protection than these 256 bits, which has way LESS unique combinations than the 4096 bits key used.

So what I really want to know is, what kind of underlying encryption scheme is used to cypher the data, because this wil determine the largest key I can use to generate the most effect.

Not quite correct but a somewhat general rule of thumb:

When the key is shorter than the bits used to encrypt: attack the key.

When the key is longer than the bits used to encrypt: attack the cypher.

This is in no way to say that for me personally the use of a secret key is inadequate or that it is unsafe or easily guessable.

And furthermore, 256 bits AES encryption with some form of block cipher-based mode is more than adequate for military applications, let alone syncapp cloud syncing.

(However if the 'encryption' is ROT13 then I'll pass )

Edited March 31, 2013 by PeterVerhees

[rdebath]

"Marko",

I've read what's been said as meaning that the limit on the amount of text that can be used to generate the key has been removed; this is good because it will increase the amount of entropy that people put into the key, making it "more random".

But very few cyphers can actually take an "unlimited" key unmodified, Blowfish is the only one I know offhand that has a very high limit (448 characters IIRC), for all the others you have to preprocess the key to "distil the entropy". The normal way to do this is to take a cryptographical hash of the entered keytext to generate the real key of 128, 160 or 256 bits. (NB: Many people forget to add salt to defeat standard rainbow tables at this point)

Based on what has been said and the size of the "info hash" that would be generated for DHT I expect the size of this distilled key is "only" 128 bits.

A "birthday attack" needs a parallel matching of all keys previously generated against the new key. This is a natural feature of the DHT network, it will, it must, automatically match a value derived from your secret against the same value from every other secret in use. So a matcher for the birthday attack is being handed to the attacker. I don't think that there will be enough keys in the DHT network to make it worthwhile, but someone else might have a better idea and this might give them a reason to make what looks like an attack against the DHT network.

OTOH: If the key distilled and used for the encryption is significantly larger than the 128bit 'info hash' the hash will not be unique enough for the for the DHT match to give you anything useful about a secret key match and there wouldn't be any reason to attack the network.

For this reason I hope the extended key is distilled into a 256bit key which is then hashed (with a constant string) to generate the 128bit "info hash". Then you don't have the one to one (approx) mapping between the "info hash" and the secret for the secret share.

Of course, as Peter said, this assumes the password isn't the weakest link. A cheap, multi card, GPU cracker (if one can be used) can test a billion keys per second, any six character password is insignificant against this level of attack. In addition the natural "birthday attack" of the DHT network means that the chance is about 1 in 300000 that two groups will accidentally get the same random 6 character password. This can easily get reduced by some sort of population stereotype. If the GPU cracker cannot be used a special rainbow table linking the (public) "info hash" back to it's secret would be workable for a password this short even with the 256bit intermediate keys.

You do need a much larger password, but you don't have to go overboard; a random password like "correct.horse.battery.staple" is probably sufficient. Though I might double it up until I'm certain that the network can't be used for a malicious birthday attack.

Peter,

But, as you say this is all conjecture, until it's been peer reviewed, you're probably best assuming that ROT26 has been used for something important

[JoelM]

I was not sure about the security of the secret too, but then I did some math. Sync uses or did use (I don't know what it currently uses) Base32 encoding. That encoding uses a-z and 2-7 so you have a total of 32 characters to use in the secret.

Small example:

The secret is limited to 1 character and there is only one shared folder (only one secret is in use, the others are not used). So there is a 3.125% (1/32) chance of collision. It will take up to 32 guesses to find what secret is in use; or 0.640 seconds (based on 20 milliseconds per guess).

Those odds sound pretty bad.

Real world:

The secret is 32 characters. Let's assume that there are 60 trillion shared folders (60 trillion secrets are in use, the rest are not in use). The total number of possible secrets for 32 characters is 1,461,501,637,330,900,000,000,000,000,000,000,000,000,000,000,000. The chance of a collision with any of those 60 trillion secrets is 0.000000000000000000000000000000004105%. Wolfram|Alpha says it will take up to 1.1*10^15 of the age of the universe (14 billion years) to guess one secret, not a specific one.

Thus to guess one random secret it will take up to 15 septillion, 437 sextillion, 700 quintillion years.

TL;DR:

Basically if every person on the planet was sharing 9,000 folders and trying to guess another secret, then the sun would be out long before one person got someone's secret.

(BTW I do not believe that the universe is that old; I'm just using the well-known information as a reference.)

[eseelke]

Sorry, I only use Linux as my OS. ;-)

No such option in the WebUI for Linux. In the config file however I found this....

There is an option, just click the gear icon next to your share in the web interface!

EDIT: Sorry, I didn't see the 2nd page or realize this was an old post.

[Xanza]

The problem with secrets is that you can "crack" them without trying. It's perfectly possible that two users can end up with the same secret and thus see eachothers files. Ofcourse the chances are slim, but they do exist. Rather than creating stronger secrets where the collision chances are reduced I would like to see a system where there are two secrets and you need them both to access files. The first secret is generated by Bittorrent and is checked against a database to make sure it's unique. The other is user generated and doesn't need to be unique. Same as a username and password except the username is just a random string.

Not only is this information wrong, it's kinda stupid to say that you can 'crack' these secret codes, but the more secure option would be to store said keys in a single location... lol If you would have said 'guess' these secret codes, then yes, I'd agree with you. But considering we're unsure yet how the secrets are generated, that's an improper assumption.

Not to mention, if the system uses something as ubiquitous as unix time-stamp for example and a variation of other box specific notation, it would be neigh impossible to decipher the secret to begin with.

We are going to change the way Secret works in next build. We will:

- remove restriction on the length;

- will use base64, so all bits will be used

- will introduce one time passwords to safely pass secret over insecure media.

I genuinely support the idea of a private, and a public (or secret) key, or if you fancy the wording better "Two secret keys." Base the key generation off of two separate algorithms, and off of misc or garbage information, and it would make even a single key very difficult to replicate. It would also ensure that someone can't simply generate a gargantuan list of possible secret keys and test all of them for valid secret keys, thereby stealing everyone's files.

[Automatic Coding]

Please no! Not another service/application depending on Google knowing what we do and when with whom and where. I LOVE the simplicity of the shared secret. (maybe certificate based sharing is a nice upgrade).

By the way, is there a way to prevent Syncapp to even connect to the bittorrent servers?

/paranoia mode ;-)

I don't think it'd be required, it'd be optional. Nobody in their right mind would say "You HAVE to use an external authentication system to use our software, even if the files you are sharing aren't really private/etc"

[PGTips91]

To prevent any collision from happening, all that is needed is to prefix the generated random key with some unique information, such as the computer name and the date/time. Then a random collision becomes impossible and a brute-force attack mandated. With longer, variable-length secrets and broader selection of characters, the chance of such an attack succeeding are sufficiently remote as to be safely ignored in most situations.

Paul

[LiquidOne]

Couple of possible options (some of which may have already been mentioned)

1) variable length secrets with a minimum length say 21 to 255

2) throttling somehow the frequency new secrets could be attempted (brute-force issues)

3) while using a shared secret manual approval of new node members by IP

4) logging for each host which syncs against...timestamp, ip, bytes transfers+direction

5) dual-secrets each side has it's own secret and negociates the pairing based up the dual-secret match ala pgp-styled

6) while not perfect geo-location...ie if I live in Australia, I don't want IP's from any other country syncing.

7) ISP based....ie if I have Comcast and the destination has Comcast we are ok, but if it's any other provider well you know what to do.

8) IP-based If I know my destination public ip is x.x.x.x/24

Outside alternatives

1) host-based firewalls, force the app to run on a specific port all the time and only allow access to the world on that port if the outside world is coming from x.x.x.x/24 or pair it with a dynamic DNS provider of your choice to lock-down the outside IP's.

2) block internet access and run site-to-site VPN tunnels.

Long term if made available via API someone could write a wrapping application/plug-in to do some or all of the above.

[GreatMarko]

Couple of possible options (some of which may have already been mentioned)

1) variable length secrets with a minimum length say 21 to 255

4) logging for each host which syncs against...timestamp, ip, bytes transfers+direction

These are already present in BitTorrent Sync - variable length secrets are supported, and the "History" tab logs each transfer

[PeterVerhees]

Did the math too, it's fairly simple.

Considering a truly random entropy with random bytes (these are not only letters and numbers you can type!), the optimal keylenght to support the 256bits AES encryption, without overdoing it, is actually 32 bytes. A simple 256 divided by 8 gives you this.

When considering that bittorrent sync uses truly random bytes in their implementation (/dev/urandom) , this matches the perfectly with the AES 256 bits encryption used. (Not debating whether random functions are truly random, lets assume they are... http://www.random.org/)

Therefore, it is unnecessary to provide extra key length. It is nice that btsync supports lengthier keys, but for the sake of the encryption it is not necessary. It might give you piece of mind....

Re-using a part of the key (without adding length) for a date or computer-name actually makes the key LESS unique and thus more guessable for others. (Counter intuitive I know, but thats the truth....).

Adding computer-name or date to the length of the 32 byte key makes it MORE unique, but adds no security, due to the length of the key surpassing the entropy of the encryption used.

See my previous posts about when to attack the key and attack the algorithm.

I myself am more than happy with the solution provided.

Generate your own key like this on Linux.

dd if=/dev/urandom bs=32 count=1 | base64

Change the bs value for lengthier keys...

(Edited, multiple embarrasing typos)

Edited April 24, 2013 by PeterVerhees

[PeterVerhees]

For true cosmic random goodness try this:

wget -qO- 'http://www.random.org/cgi-bin/randbyte?nbytes=32&format=f' - | base64

Change the 32 to someting else for more or less bytes.

[splat]

rdebath: Excellent analysis. As I understand it, the birthday paradox is the fact that collisions will happen in a uniformly random distribution more quickly than the (naïve interpretation of) the number of permutations would suggest. This becomes a problem when the width of the distribution is used in lieu of the actual collision probability when proving security.

That said, with the current 21-byte width of secret keys, the possibility of an accidental collision is nearly impossible. Assuming new keys are created at a rate of 100,000 a second (meaning every living human is adding a new sync folder daily), there'll be roughly 450 universe ages before we have to worry.

So the question becomes: can an attacker beat 100,000 hz, keeping in mind that the birthday problem only applies to finding any collision (so the attacker gains access to some network) and not finding a specific collision (gaining access to a specific network), which still requires brute forcing the entire keyspace.

I think the distributed nature of the network helps quite a bit, here. No global view of every existing secret exists (save, perhaps the set of trackers, but I'll come back to that in a moment), so in order to check my guesses against the previously generated secrets I have to physically transmit them all over the earth. That introduces a lovely bandwith-delay product limit on how quickly I can guess and check; some back of the envelope calculation suggests that I'll need a few orders of magnitude more capacity than exists in the internet (but don't take my word for it -- I hope that someone else will do the math and end up with a similar result).

So, my only hope is to localize the guess & check as much as possible. Depending on what kind of state the trackers keeps as they mediate connections, this may be relatively easy -- if I can break into a tracker and steal its state, then I have at least a partial view into the network-space and a easy local guess & check algorithm.

My understanding is that the trackers keep sha1 hashes of the secrets, which is unfortunate. If I can compromise their data structures, I can use my GPU to check billions of guesses per second and the 2^84 hashes I'll need to compute on average to find a collision doesn't seem quite so large. It's still significant, presuming the keys are generated by a secure pseudorandom number generator, but Moore's law stops being our friend. Happily, this is a solved problem; the tracker should just use bcrypt, which effectively introduces the per-guess latency locally that the internet gives us for free.

Or, using 32 bytes of entropy instead of 21 means, even in expected random guesses until collision terms, there's a roughly 2^-128 probability of a collision.

But, as you say this is all conjecture, until it's been peer reviewed, you're probably best assuming that ROT26 has been used for something important

This, I think, is the most important point of all. Kos et al, are you planning on publishing the specifics of the secret mechanism (or, better yet, the code)? I'd also be interested in the details of how the inter-node communication encryption is handled, but that's a problem for another day.

[splat]

To prevent any collision from happening, all that is needed is to prefix the generated random key with some unique information, such as the computer name and the date/time. Then a random collision becomes impossible and a brute-force attack mandated. With longer, variable-length secrets and broader selection of characters, the chance of such an attack succeeding are sufficiently remote as to be safely ignored in most situations.

Paul

Paul, that doesn't prevent (accidental) collisions, it just means that two users have to additionally collide on name and date/time. The entropy of that information is actually relatively low per bit (<1, since you and I will end up with the same 5-byte computer name far more frequently than we'll generate the same 5-byte random string). We'd stave off more collisions by taking the same number of bits used to prefix with that low-entropy information and replacing it with high-quality random bits, buying us a full bit of entropy per bit of storage (assuming a secure generator with a high-quality source).

It certainly doesn't help prevent attacks, either, since computer name and date/time are also extremely predictable. Again, if we're allocating that much more keyspace, I'd rather use it for purely random bits.

[kos13]

My understanding is that the trackers keep sha1 hashes of the secrets, which is unfortunate.

It is SHA256(secret).

We plan to expose all security details of the Sync.

[splat]

1) variable length secrets with a minimum length say 21 to 255

As a general rule, users cannot be trusted to compute securely random strings of any length. Plus, as PeterVerhees points out, having a secret longer than 32 bytes doesn't help much -- and 32 bytes of entropy is plenty.

Allowing users to pick their own keys (and lengths) merely adds to the feeling of security while reducing the reality of it. That's what Bruce Schneier calls "security theater." I'd much prefer fixing Sync secrets to 32 bytes, preventing users from generating their own, and publishing the details (or the code) of the internal generation process for review.

[splat]

It is SHA256(secret).

We plan to expose all security details of the Sync.

Wow, thanks for the speedy response! I'd strongly encourage you to use bcrypt(secret) rather than SHA256(secret), as the former is designed exactly for this scenario and the latter was designed for a different purpose (message authentication).

Also, side-channel attacks being what they are, OK for me to read "expose all security details" as "release the code"?

[kos13]

We don't like "security through obscurity" approach and will expose all details. We also understand importance of the implementation, so we will do something.

[LordVonPS3]

I'd like to make the following suggestion.

BTSync should require a 2nd secret (e.g. 32+ chars complex string or perhaps a simple 6 digit PIN code) when first connecting to an existing secret / client pool and impose a 5 attempt limit.

Failing to enter secret #2 correctly five times would result in all secret #1 sharing clients adding the requester's IP address to an indefinite blocklist for that shared folder / secret #1. To facilitate this, some hidden, plain-text blocklist file could be shared between clients, meaning any client(s) could remove an IP (or all IPs) if some mistake were made.

[rdebath]

I think once thing that should be highlighted here is the difference between the length of a key and the amount of entropy in a key.

Consider these three keys

1. DOL5OMDZBXP5CX7FLRWOJVITKTNJYL2O
   
2. R27WAH4LQCGDFOGS7NLQYLQPXW5TRCW5
   
3. OO0O0O0O00OOO0O0OOO00O0O00O0O00O
   

They are all 160 bits long base32 encoded.

The first one is all that a key should be, it was generated from a megabyte of internet network logs so is basically guaranteed to contain the full 160 bits of entropy.

The second one looks exactly the same; but convert it to hex and Google will find you 250 hits ... it's probably got about 10bits of entropy.

And as for the third one; it has no entropy even the 'O' '0' mixing is zero, because 'O' and '0' are the same place value for base32.

If BTSync is going to use the nice simple shared secrets (Never passwords, always pass-phrases) you cannot stop the users choosing bad ones. All you can do is give them lots of opportunities to choose good ones. Give them several boxes to full in, pre-populated with the machine name, date, user name and random string. Anything to increase the amount of entropy.

Take the SHA hash of all of these together for your secret; stick everything together to make a base64 blob to copy & paste between hosts. Make the boxes BIG! Let them drag and drop a file onto dialog; take the hash of the file and add it into the random box.

Really your only other option is to go the full certificate route where every node has prior two way communication with a trusted third party. That's not simple (and conflicts with several use cases).

[rdebath]

adding the requester's IP address to an indefinite blocklist for that shared folder / secret #1.

BTSync uses UDP, the source address of a UDP packet is trivial to fake.

This would make for an easy DOS attack on a share.

Also, a second passphrase is logically part of the same overall passphrase, splitting it only makes the attacker's job easier; see the netbios 7/14 character passwords for a notorious example.

Furthermore, it's usually reasonably easy to evade an IP block.

[affinity]

The fact that any new secret creation can cause a collision is enough to need some other factor.

I don't want Google Authenticator, which by the way, can be fully implemented WITHOUT using ANY Google servers. The code to use this kind of time based one-time-password is known, in fact I use a plugin with KeePass to generate TOTP codes when required for Gmail and other services, rarely do I bother with the GA app on my phone for the code.

If you have the 16 character string that is used by GA, then you no longer need Google at all, in fact, even if you do use the GA app, then the generated key is done on your device without communicating with Google (if the app does actually communicate with Google, then it is an "extra" that isn't required to satisfy the requirement).

Definitely, this post from the other thread indicates what I think will be the best way to deal with ANY possible collision (not talking at all about brute-force here, only accidental collision(s).

http://forum.bittorr...__40#entry48294

This too:

[wimbit]

Question: Bit Torrent Sync does not know my secret, they just know the HASH of my secret, right? SO, if they are compromised, will anyone ever know my actual secret?

Also, given that someone *could* guess my secret or collide with it. Not likely, but possible.

... If I generate a Billion 20 character (a-z,A-Z,1-9) words (secrets) how many of them will contain my email address? If I preface my email address to a secret, then how much less is the probability that someone would find my secret randomly?

Any math wizards want to take a try at solving that ...

[Heavensrevenge]

Not meaning to revive a dead thread but if security seems paramount, Sync should just use the new SHA-3 Checksums of any length anyone wants. The Algorithm is named Keccak and is the new SHA checksum blessed by the NIST So yea you should completely rock Keccak for checkcums or secret creation in some future build to really be ahead of the curve in terms of security.