mreithub

Hi all of you, first of all thank you for all the responses so far (especially for the ones ) When I posted that message I already suspected that some of you might not be all that happy about what I am doing... I've been thinking a lot about it, but I think I've done the right thing. And what I've done so far was basically a mix of thinking how I'd do it myself, running tcpdump+writing a script that makes it easier to see patterns in the transmitted packets and a little research in "[uses] bittorrent technology", so it hasn't been that difficult (and would've been done anyway sooner or later). I am in no way trying to pose a threat to btsync. It's in my best interest that it'll be as successful as it can be, because I am (as most of you I suppose) part of the minority of cloud storage users that really are concerned about having their private data stored unencrypted in some data center (we've all heard the PRISM discussions lately, so I'll skip that here). But as I said, I think we (the security-concerned users) only make a small percentage of cloud storage users. So for btsync to gain real momentum, it has to be rock solid. They say they use secure AES-based encryption (which I believe is correct but I (and I'm sure many others) would prefer to have actual proof. The other killer feature of btsync - the (virtually) unlimited storage - comes with a downside though: It's only interesting for people having their own server or at least NAS with 24/7 uptime. There already were proposals in the forum for an additional secret that'll only gain you encrypted access to the share. That was also one of my first thoughts after learning about btsync as it will allow us to get rid of the one downside and allow btsync-providers that can't access our data. Those could also implement daily off-site backups of your encrypted data - which will be a very interesting feature for small and middle-sized businesses (which in turn will increase btsync's market share). I do want btsync to get as successful as possible (as I don't want to fall back to dropbox and the like when it comes to sharing data with people that simply don't have it) so market share is important (and the market's already close to being saturated if you ask me) As to your statements that it doesn't make sense to analyze a protocol that's not yet stable I can say that I think (like @cdata, ...) you can't start too early for several reasons. The first one is that one of my aims is simply to prove (or disprove) make sure the used encryption really is secure (All the threads @GreatMarko linked to are either about implementation bugs or "how-secure-is-the-secret". That's one part of the question, but I'm more interested in things like: which AES-Mode is used, does it encrypt everything or just the files, etc.). The other reason is that I look at BitTorrent itself and the huge ecosystem of apps and services that grew around it. I'd like something like that for btsync too. I also see other services built based on the same technology (e.g. think of having a remote desktop app that uses btsync technology (not the file sharing, but the discovery and transport layers)). To sum it up, I'd love to see other developers using that technology for whatever reason in their apps (there's just so much potential in the technology). To the API: Yes, I've read the announcements of an API, but as @rippelhans answered to the post @GreatMarko linked above, it kind of pokes a hole into the whole noone-else-has-access-to-your-data argument. Yeah, they could write an API-Software that's self-hosted or runs on the client but then you're back at needing a server. Ok, that about sums up what I wanted to say. Sorry that it's quite a lot to read (and sorry if some parts may be hard to read, I'm not a native speaker). Anyway, please ask if anything's unclear or I left questions unanswered. (And help is very much welcome )

I've been spending some time recently trying to figure out some of the details of BitTorrent Sync's network protocol. I really liked the idea of a p2p alternative to other cloud storage providers like dropbox etc. So I tried to find out the technical details behind the project. The official Technology page was a little sparse in that regard so I figured I'd have to dig in myself. I managed to start this as a university project (at the Vienna University of Technoligy in Austria). The goals are: Analyze the network protocol (and documenting it as far as possible) Discuss possible security issues (The official page claims it's very secure and throws around with words like AES and SHA2, but ) Don't get me wrong, I don't expect to find anything serious here as the basic idea seems pretty solid, but I think this claim has to be proven Write an open source library that allows other apps to communicate with btsync (not a full-blown sync app, just the basics) So, now to the fun stuff, I'll explain what I've found out so far: Secrets: The read only secret is basically just a hashed version of the normal secret (all code is valid python code (after importing hashlib.* and base64.*)): roSecret='R'+(b32encode(sha256(b32decode(secret)).digest())[:32]) The roSecret is hashed again. That hash is transmitted in LAN announces as well as to the tracker at t.usyncapp.com: shareHash=sha256(b32decode(roSecret[1:])).digest() That's about all the magic that's in the share secrets. Tracker: The Tracker protocol uses bencoded data. The request format is (I've conveted it to JSON for readability): { "peer": /* 20 bytes of binary randomness identifying this app instance" */, "share": /* 32 bytes (=256bits) SHA256 hash, see the Secret wiki page linked below for more info */, "m": "get_peers", "la": /* local IP + port in binary form (4 bytes IP + 2 bytes for the port) */ } The tracker's answer looks like that: { "m": "peers", "peers": [ { "a": /* 6bytes external ip+port of peer*/, "p": /* 20bytes peer id (random) */, "la": /* 6 bytes local IP+port of the peer */ }, /* all the other peers */ ], "ea": /* 6 bytes, external IP+port of the local peer *, "share": /* 32bytes shareHash */, "time": /* integer, unix timestamp */ } P2P: The peers use µTP for communication. On top of that, they seem to do a key handshake and submit the actual data. The details about that are still a little unclear (and have changed since the early 1.0.* releases), but I'll keep you updated as soon as I know more. I'm currently working on some Qt code (Qt's just the framework I'm used to, this might change if there's a good reason to) that reflects these protocol details. It's mostly based on what I've found out analyzing v1.0.116's protocol. I'll update it soon to support the new protocol used by 1.1.27. It can be found on GitHub. All the above details also are in the repo's wiki (feel free to contribute). Also let me give huge thanks to the BitTorrent Sync Team. They're doing great work (both in writing the app and in working with the community here on the forums).

@GreatMarko Yeah, I know that, but I'm thinking of users who want to sync their data on the go with changing IPs. I think of my proposal to be just the icing on the cake of privacy btsync provides. And it wouldn't make life harder for those users who wouldn't use this feature.

Hi, I've been thinking a lot lately about whether it would be possible/useful to remove (or better: to give users an alternative to) the one centralized infrastructure btsync uses: the tracker. Currently, the app registers its shares (and asks for peers) at r.rsyncapp.com and/or t.rsyncapp.com, but what if we introduced an (optional) URL scheme for the secrets? SECRET btsync://SECRET btsync://tracker-server/SECRET The first one for compatibility (and simplicity) and uses the official trackers The second one is basically just like the first one but with a URL schema The third one might use another tracking server (i.e. a local one in a company) While I don't think this will be used a lot, it's a simple extension that might give those who don't want their IPs logged abroad (the tracker might be used to store an IP history for each peer) an extra feeling of privacy.

btsync might have a problem locating its web interface files (just guessing, I get that error when entering a wrong URL to my browser, like: http://localhost:8888/abc ) Your browser goes to http://localhost:8888/gui/ right?

Indexing the big files might just take a little while. I don't know what btsync indexes exactly, but I think it's generating some sort of checksum of chunks of the file (like bittorrent does...)). So it has to read the whole 175GB of VM image files which might take quite some time. Btw.: I don't quite know whether btsync is the best software for your purpose: If VirtualBox is running, it writes to the virtual disk quite frequently, with each of the writes causing the file system watcher to fire and btsync to checking the file (and trying to transmit the changes). A better idea might be to setup a daily backup process... But I don't know the internals of btsync, so please if anyone notices anything wrong here, correct me

From what I've read in other posts it seems like you don't want to open-source the btsync app itself. But are there plans to provide a protocol specification (such that devs could write their own apps that communicate with the official clients)? This would enable an ecosystem to grow around your official app like it happened with bittorrent.