lolcat

BTSync can probably not be open-sourced. If BT inc were to open-source it they would have to release the code for µtorrent. The protocol can be documented though. The worst part of this is that a large portion of their code is allready open-source. Sqlite, openSSL, google crashpad, and so on. Replace µTorrent with libTorrent and then the rest of the implementation would be pretty small

What kind of specs does the remove server have? Does the remote server use an encrypted read-only secret, a read-only or a read-write secret? What kind of harddrives does your server use? Is the I/O limited? When seeding a file with a read-write secret the server has to read file -> encrypt -> put torrent pieces into SQLite database. This database I am sure is limited in size. This means seeding one file involves reading the file (using read IOPS), encrypting it (using CPU cycles), adding it to the database (write IOPS), and then reading and seeking the database to serve the files (more read IOPS). The encryption probably requires some ram too. I think BTSync caches most of the database in ram (I hope so, it uses tons of ram that can be swapped out, stupid).

You're welcome.

I am assuming you gave BTSync the encrypted folder? If so then the answer is pretty obvious. It is a normal folder, files and folders written to it won't be encrypted. The FUSE mounted folder on the other hand, it works as a transparent layer between the encrypted files and the folder you see. This is not a normal folder, the folder containing the encrypted files is a normal folder, it can be written to directly.

This statement made me lose faith in humanity. Please excuse me while I shove a rusty tablespoon into my brain. This is by far not "the most secure thing ever". I wouldn't even call it secure. I would assume everything is leaked. The encryption scheme isn't documented, or reviewed, so it offers no security. The program is not made for anonymity, so it has no advantadge there. The only difference from storing it on DropBox and using this is that the files is not stored on third party severers with intent. Ofcourse if you worry about the NSA they will likley be able to download the datastream, ofcourse if you assume the encryption implementation is sound (and that is a pretty fragile assumption, almost like assuming Titanic can't sink because the creators said so) then they can not decrypt the data. But they could retain the data forever, so if you use the same secret for different stuff, then they can decrypt everything you shared before. Using the tracker adds extra vunerabilities, then the NSA can get the IP of every node connected, they only need one person with a read-only secret to decrypt everything sent with that secret, and to decrypt everything that will be sent with it in the future. So to sum it up: BTSync offers worthless encryption, no anonymity, and the files are easily intercepted between the nodes. In addition it offers no forward secrecy. If you want something secure I would recommend you to encrypt the files using AES256-CBC, using different keys for each file, and then setup a SSL Torrent using LibTorrent. Then you can be assured the keys are properly generated, ensure access control, and you will have forward secrecy.

These files are created by BTSync. .SyncArchive contains deleted files, .SyncID I assume identifies what share it is part of. And .SyncIgnore is there to tell it what files to ignore. I don't belive any of them are actually synched. (and appart from syncID none should contain anything if configured correctly).

Wireshark where? The person receiving the files can see the connections to you. Anyone inbetween can see traffic between you. Bittorrent Sync is completly unsuitable for anonymity or confidentiallity. You could set up a server with the encrypted key, and then connect to it not using DHT, the tracker or peer exchange. Then the downloaders will only see the IP of the server. Obviously you have no guarantee the application doesn't leak info to Bittorrent Inc or others. Sure, if you completly trust that setting, and assume the IP won't be transmitted through any other feature.

To secure Sync I would recommend you encrypt the files yourself, the Bittorrent Sync team has earlier changed the encryption scheme without warning the users (as long as it is a beta that could happen). Everyone downloading the file will see your IP-address, as will Bittorent Inc as they host the tracker. You can hide your IP by using a VPN or a proxy.

Version 1.2.82 uses a lot of ram on FreeBSD. It used like 700MB of swap, that is like, more than all the other things (including the OS) is using. I hope it isn't caching its databases in ram as teh filesystem will do that automagically. What is it using all the idle ram for? If it is getting swapped out I don't see the purpose.

Use Pidgin or any chat client with OTR-encryption?

Would you be as kind as to offer the sourcecode for your application? I would love to be able to host something similar on my box.

FreeBSD 9.2-RELEASE FreeBSD 9.2-RELEASE #0 r255898: Thu Sep 26 22:50:31 UTC 2013 root@bake.isc.freebsd.org:/usr/obj/usr/src/sys/GENERIC amd64 That is my "uname -a", and it works like a charm. So I would say, yes, it does. Just download the binary to a suitable location and run it. There are scripts to start automagically at boot and stuff. Google should be able to locate them.

Which one of those isn't self explanatory? First one it can't find the folder at all, second one the contents of .syncID isn't what it should be, third one it is on a network share and you should configure it using the drive letter (D:, E:, F:).

The secret keys are handled by an external server? Where did you get this idea?

Isn't one way sync as easy as adding a read-write secret on the phone, and a read-only secret where you want to download it? Move the old files to a folder elsewhere on the phone, sync the empty phone camera folder.

Nested shares could also accomplish this, in a far more secure way. Protecting the tracker would solve nothing, the other person could still just type in your IP:Port and download your files. To continue to encrypt files with a compromised key is also a bad idea. The files that are already shared with that person is rather irrelevant as they will already be downloaded. But the new files you would want to be encrypted with new keys. This would require encrypting files with their own unique key, and enable lots of fun possibilities. You could then share individual files, build a share using any file/folder from your share, and enable cross-seeding. If you then attached your "virtual" share to a different share for each person, all you would have to do is to detach the share from the share the person you don't wish to share with, and they would lose access to all new files. You could probably do this without restricting their ability to seed/download the files they already have.

Tahoe-LAFS is far harder to setup, and is also a distributed central system. The advantage of BTSync is that I can share files with people who can then also share it. I can't install tahoe-lafs on my grandmothers computer, I can't set it up and leave it, and I can't have her seed the files she has downloaded. The advantage of BTSync is the p2p aspect. I can share a massive amount of data with ease, and the more I share the more available it will be. If I share my photos with my grandmother, I can download them utilizing her bandwidth. If the encryption scheme was changed it would make managing this so much simpler. Tahoe-LAFS would probably be a good solution for OP, because his solution already removes the biggest advantages of BTSync. Nested-shares, and being able to create virtual shares using pieces of an existing one would enable far more complex distribution models, while still getting the full advantage of P2P. It is the difference from having an easy way to send a limited set of files to a few users and the ability to send massive amounts of data with multiple people. Having one share with ALL my files, and then being able to tailor shares for each person while still enabling cross-seeding will increase the size of my swarm. If I used OPs setup I would have to seed every file with every person I want to share with. Then what would be the difference to making a centralized setup with a DNS setup that makes people download from the server with the most free resources? And he has also removed the ability to use encrypted read-only secrets to store his files on insecure locations. It also requires that he has full access to the servers hosting the files to create the hard-links. The good thing about BTSync is I could store my files on my friends desktop, without him seeing the files, and without me having any access to his desktop. If it was done using my solution I would simply give him an encrypted read-only secret, attach the files and folders I want him to store and distribute, and then I'd have replication and additional bandwidth using his spare space and bandwidth.

Yes, he has effectiviy turned p2p distribution into a traditional model of centra content delivery. When each person has its own share, they can't seed to eachother. This is fine if his bandwitd is sufficient to share everything. But then what is the point of BTSync? Some dropbox-clone with a webUI would do pretty much the same, the only advantadge here is the resume option for huge files. Implementing this with encrypted read-only keys also seems like a pain. If we had nested-shares and different encryption for each file and folder we would finally be able to setup shares from a massive share. Then you could make a share containing the right files and folders. And all the users would be able to cross share. This would maintain the p2p abilities. If we also could split a massive share into pieces we could store it across several nodes. Think if you could split the share into 1GB pieces wtih different offsets. 4GB would then create 8 pieces, the first four starting at 0-1gb, 1-2gb, 2-3gb, and 3-4tb. The other four starting at 500mb-1500mb, 1500-2500, and so on. This would be good for when a node goes down, if 500-1500mb goes down, the load will be shared across two nodes rather than one.

Is your encoding borked or is it like that in the log?

Great work, this seems incredibly useful. Does the peers only announce the share to the tracker? Is any other information leaked to the tracker?

I assume you are using keys as a synonym for secrets, is that correct? What additional "encryption information" are you referring to? If I want to send you one file, then I need to encrypt it. If the torrent is made using unencrypted files, and then before the hash checking the file is decrypted, then I would be able to share a massive folder with no more overhead than the hashing for the torrents. For the encrypted read-only this approach obviously won't work, as it would have no way to check the files integrity, because it can't be decrypted. Is this the problem? If that is the case, you could have to encrypt every file and folder name, and then make the torrents. But still, to transfer the files the aes-256ing of the files would use the same amount of cpu cycles. The only problem would be for a large amount of files where you can't store the encrypted versions (so you would be forced to first encrypt all your files, and then encrypt them again when someone wants to download them). The second part of your post makes no sense. I understand they don't have access to the encryption keys, but that is as simple as not providing them with the encryption keys. The files are encrypted to be sent (aes256) and encrypted to be stored by the encrypted read-only nodes. Is the performance issues just seeding, creating or downloading shares? For seeding there should be no additional overhead, other than maybe some meta data when you create a share (to get torrent hash of the encrypted file). Seeding or downloading would be no different, simply seed the torrents without giving up the encryption keys.

Interesting. Any idea why? I would imagine that it would increase the load only for a peer that is an encrypted peer. I imagine the only thing that could make it harder is that you would need to create two databases (one for normal peers with the file lists and one for encrypted peers with the encrypted file names). The method used for secret v2 is to create a sqlite database containing all the files. It has two tables with two fields, one is the location of the file in plain text, and the other is various attributes bencoded in a BLOB (owner, timestamps, something that looks like a torrent info hash, size, type, sig64, and so on). Each folder appears to be encrypted with its own key that is obtained from the swarm, exactly how this is done is unclear. Each file is also its own torrent. For the encrypted read-only key I assume you need another table, one where you don't leak all the information. If the encrypted read-only node could see the owner, that would be unfortunate. Maintaining these two different databases would add overhead. This blog lists the format of new keys, and creates them as you visit the page. I thought I could make my own secret using the information gathered there. I was wrong. If the new key-scheme were like the old one (A+base32(sha(whatever)) I could just copy the first letter and add it into the client. I was able to add the new keys into the client, and the new read-only secret was the same format as on his site. But I was not able to remove the extra characters from the read-only secret and get a valid encrypted read only secret. FU7BUKNYINCPE3I32Q2UTNCFJ6OS4M7TV <- Encrypted read only, the first part is the same as the read-only secret EU7BUKNYINCPE3I32Q2UTNCFJ6OS4M7TV5YP5C5ZMT767D3BV4C5NVMQ6BE <- read-only secret D6EY5DKO3EWCRIWM2XFTJLCAJMWD63A6X <- read-write secret Now if someone were to figure out how the read-write secret is generated, we could create our own secrets. Replacing the A in a secret v2 with a D, will let your client create a read-only secret with the same length, but the client won't accept the first part as an encrypted read-only secret. DJRLNSE3QUUQO6SU5ZTTIQGH5MRILCAG7 EZ3NCAAPT43XF7LPQQLE75RCWJKALAVCXTLJAUBSDXNAUNVZHGVBDV2LXLA Using the first secret (a secret v2 with the a replaced with d) to create a read-only key results in this. I haven't tested if the secrets actually works, but I am able to add them to the client. Figure out how to make a valid read-write secret, and the feature is unlocked. I think the rationale behind blocking users from using this feature is weak, simply add a warning that that kind of keys isn't suitable for mobile devices and warn them to use caution. That way daring users can play around with it.

Agreed, bypassing the limitations seems fairly simple. It seems even the webUI could easily be modified to remove restrictions of features.

[2013-12-28 00:37:24.547] SyncFilesController [file updated]: processing file \\?\C:\Users\Jo Beth\Pictures\2005\12\Resaves\DSCF1405.jpg 1388211256 2584450 [2013-12-28 00:37:24.548] SyncFilesController: missing metadata for file 2005\12\Resaves\DSCF1405.jpg, will force re-indexing 2584450 is a unix time stamp, this is almost epoch (when God invented time, around 1/1/1970). 1388211256 also a time stamp for Sat, 28 Dec 2013 06:14:16 GMT, I am assuming either your time converted to GMT (or your time is displayed in CET). If you check the file time stamps, can you read them? I assume BTSync will check when the file was edited, clearly it fails to do this for some reason. Are they valid times? Could you open them in lightroom, edit something (add a few lines of pixels or something) then save them again. Check the time it is last edited, if it is a sane date (ie after 1/1/1970), and see if they now will sync. I am sure there are other ways to edit time stamps in Windows, but I don't know how, and don't care enough to figure it out.

Unless the shares are on different drives you would still be limited to the I/O of the drives. On a VPS I'd expect I/O to be the bottleneck. For someone using 7 drives and no raid setup it would make a great difference though, but in that case I would just recommend them to use a raid. Besides there are far more pressing issues, like selective downloads and encrypted read-only secrets that are vital for usability.