PeterVerhees

[Now Implemented!] Support For Untrusted Encrypted Node.

Recommended Posts

Disclaimer: Could not find something similar as a request, however, I've also put not much effort in searching the forums exhaustively.

Example:

I've got plenty of disk space and plenty of network bandwidth and am willing to share this with my family. So I'd like to provide them with a btsync solution. However, since we in our family are all bordering to clinical paranoia ^_^, my brother does not want to store his files in clear text form on my storage AND I do not want to be able to read them. (Hell knows what he is putting on MY disks, I really am not interested in his weird fairytale fetish :blink:)

Is it possible to have a new type of secret which allows me to sync the files for him but store them in an encrypted way?

This would also be interesting for people who have some rack-space or server somewhere and want to utilize the space without running the risk of getting their private data compromised when for some reason the machine (virtual or not) falls in the wrong hands. (yes this could be accomplished with another form of disk encryption)

Anyone second this option?

PS: Check this related topic too: Quota Support

Edited by PeterVerhees

Share this post


Link to post
Share on other sites

(yes this could be accomplished with another form of disk encryption)

I think you may have answered your own question there!

BitTorrent Sync is first and foremost a synchronization utility. Whilst connections (and data transmission) between devices are encrypted, BitTorrent Sync won't process/alter the contents of the actual files themselves i.e. the file sent will be the file received.

To share/sync a file but not have the recipient be able to access/open it, the best option would perhaps be to use a 3rd party tool/application to first "encrypt" the file at source before syncing it.

.zip or .rar files, for example, offer encryption & password protection and have the additional benefit of compressing data too (potentially faster syncing!). For those concerned about files containing "clear text" being sync'd, perhaps WinZip, WinRAR, 7Zip, or one of the many other compression/encryption solutions out there could be of use to protect sensitive files prior to syncing/sharing.

I'd be surprised if the idea of having a special "secret" that would "convert" files being syncd into another, encrypted, file type at the destination makes it into BitTorrent Sync - not only is this a bit beyond the scope/purpose of a "synchronization" utility, but it would also present a number of challenges, particularly with a R/W sync - for instance, let's say on device A you have a file that you want to arrive at device B as another file i.e. in some "encrypted" format. How would BitTorrent Sync keep tabs on these two different files given that they won't then be identical in terms of name/size?

You could end up with a loop/cascade/race condition, i.e:

  • Unencrypted "File 1.txt" on Device A is encrypted by BitTorrent Sync and arrives as "File 1.rar" at Device B
  • "File 1.rar" on Device B then sync's back to Device A as "File 1.rar"
  • "File 1.rar" on Device A is encrypted (for a second time!) and sync's back to Device B
  • ...and so forth!

However, one potential solution to achieve what you're trying to achieve would be to use the command line interface of something like WinRar, and a batch file scheduled to run on a recurring interval. The batch file would automatically create encrypted .rar files of the individual files within the monitored folder.

The same folder could also be added to BitTorrent Sync, and by making use of the .SyncIgnore feature, ignore (not sync) everything other than .rar/.zip files. This would essentially achieve the automated "encryption" you're after!

In summary:

  1. Unencrypted files are placed into a folder
  2. A batch file runs and automatically creates - in that same folder - password-protected .rar archives of the files
  3. BitTorrent Sync is set to only sync .rar files from that folder (and ignore all other files)

... but again, I think it's a bit beyond the scope of BitTorrent Sync to be able to provide this kind of file encryption itself.

Share this post


Link to post
Share on other sites

I realize this is beyond the scope of filesyncing, however, encryption at the sending end is not always feasible (phones for example). My proposition is merely to feel what others might think and tank you for your view.

I do not write the software and have no idea how it works, but one thought I have is that the before an incoming file get written to disk, the decryption get circumvented or bypassed. Again, I have NO idea of the feasibility of this.

Encryption at the receiving untrusted end (with other software like truecrypt for example), still implies the receiving end can encrypt en decrypt it, that is not what one might wish.

As for myself, in such a situation I am more than happy to use EncFS of something similar, just putting up the idea for the masses.

Maybe the secret key system lends itself easily for this.

Edited by PeterVerhees

Share this post


Link to post
Share on other sites

I'll second this idea. Just because I trust a machine to have some level of availability should not imply that I trust whoever owns that machine with my data.

Share this post


Link to post
Share on other sites

@PeterVerhees

I also like this idea. I'd like to be able to use some spare space at a friends house as a secure store. Understanding how BTSync works and the issues with socket based encryption vs. file system encryption - I doubt that this would be that easy for them.

I'll put a vote for it none the less.

I know that you want to avoid encrypting at the sending end, but if you wanted something that worked right now you should be able to have your brother point BtSync at a TrueCrypt volume and then send you a read only secret. My understanding is that BTSync already supports sending deltas from changes in a TrueCrypt volume.

Of course if BT do decide to support something like this you would want to be able to set a file size quota on a share so that they don't hog your entire drive.

Share this post


Link to post
Share on other sites

What you essentially want is a backup solution. You can try something like CrashPlan which is free if you store on your own / friends’ devices and also encrypts the backup.

I think that the BT team should focus to make the current version more reliable, less resource intensive and more user friendly. Feature creep just hinders that. The focus of the application is still sync, i.e. before getting 20 features for very specific use cases, I prefer to get a simple, stable and fast folder / file sync (that also transfers HFS resource forks like Finder labels and handles symbolic links). They are working hard and the product is good at the moment, even in the alpha / beta phase, but there is still a lot to do to get the currently implemented features right. A great first public release with less features is better than a half-assed product with 100 features.

Share this post


Link to post
Share on other sites

What you essentially want is a backup solution.....

....half-assed product with 100 features.

No, I am not seeking a backup solution. I just want to share my bandwidth and storage so my family does not have to leave their computer on at all times if I already am leaving my NAS on. (Sync to phone and or laptop etc etc...)

Furthermore, BTSync is such a beautiful product just BECAUSE it syncs live data and not provide a backup only solution. I do not want to store my data anywhere but at home. My family trusts me more that any other third party (at least they have not led me to believe otherwise smile.png). It's just that when files get synced to my storage I have plausible deniability when I cannot access the files due to encryption.

And hey, it was just something I thought about as a "nice to have" and in my case this specific nice to have is more important than HFS or HFS+ resource stuph or finder labels since I do not use those.

You reason from your point of view as I do from mine, but calling it "half-assed" because of someone liking/wanting some functionality is a bit strong, don't you think? huh.png

Edited: because I did not read the Crashplan line thoroughly...

Edited by PeterVerhees

Share this post


Link to post
Share on other sites

That’s the way software development works. When you add a ton of features you won’t have enough time to make them great. When you make something great, you won’t have enough time to add a lot of features. I didn’t mean to insult you, I’m just convinced that keeping the current functionality will be sufficient for 80–90% users. I’ve also some very obscure workflows, but screw me and others who are similar. I can write my own scripts or wrappers that make use of BT Sync. I see support for file system metadata or symbolic links not as a new feature, but as an improvement to the current file synchronization.

Furthermore I don’t think that I understand your use case and your clarification just confused me. Can you please explain what exactly you want to do with your family / brother and probably give an example? There’s probably a workaround for your feature request and I’m glad to help.

Share this post


Link to post
Share on other sites

There is a nice workaround already called EncFS which I personally use to great length to store data elsewhere (even on USB disks or sticks I personally own) and is usable on all platforms.

But this is cumbersome, just as cumbersome as zipping, rarring or tarring files to preserve file attributes or metadata before putting them on BTSync. It would be nice to have this functionality in one package so it is transparent to the "end-user". Using archiving defeats the "live sync" functionality.

Thanks for your interest thought and your offer to help.

Use case:

My bother has a newborn and took pictures of the birth. Really close-up personal, not to be published anywhere, but for him and his wife VERY important pictures. If you catch my drift...

I am more than willing to store them on my NAS (which gets backupped to a second NAS with de-duplication AND versioning) which provides piece of mind he himself can never achieve. He is not techsavvy.

However... I can fully understand that while he trusts me not to open the pictures and me telling him I have NO interest in them AT ALL, he still would like the assurance of "end-user" controlled encryption to block me from seeing this at all. Not being a techperson himself, it would be nice if the software support it.

I hope this clarifies things a bit, I can be a bit fuzzy when trying to explain. unsure.png English is not my native language...

Share this post


Link to post
Share on other sites

@PeterVerhees:

If you want to be able to control everything from end to end, believe me, what you need is EncFS.

It works well with Goolge Drive, Dropbox and similar solutions, so I guess it also works with BTsync out-of-the-box. (I'm rather confident to mention it here as I'm currently testing these three solutions in parallel: BTsync, AeroFS and DropBox with EncFS.)

If you're using a Mac, you may be interested in reading this article which provides an easy EncFS installation procedure (DropBox specific, but should work with any other similar service).

But if you (or at least your brother) don't want to get your hands dirty and if you are not too paranoid, you may be interested in some form of distributed + zero-knowledge encryption cloud storage. Take a look at Symform, LifeStuff (public alpha soon), or even Space Monkey (a project on KickStarter).

As long as I'm concerned, I'm a big fan of the K.I.S.S. (Keep It Simple, Stupid) principle. And I'm not sure that adding data encryption to BTsync is a good idea...

I hope this helps!

(English is not my native language as well.)

Share this post


Link to post
Share on other sites

I've been thinking about this some more, as many people seem to be requesting it. It does feel a bit like it would bloat up and complicate BTSync, but I still like it. I think, though, that the reason it seems to fit here, and not so much with Cubby or AeroFS, is because BTSync runs on NAS devices. That kinda opens the door for the 'shared' space scenario.

If they were to implement it, it would need to do all the encrypting on the client side - you don't want your friend/brothers keys ever on your side of the net for security reasons. BTSync would also have to manage encryption keys (client side), couldn't just use the secret obviously. From a user interface point of view, maybe BTSync could generate an encrypted / storage only key similar to how they generate a RO key today.

Share this post


Link to post
Share on other sites

This would be moderately difficult to implement IMO, but it's not part of the 'sync' process at all. It would need an extra passphrase independent to the SyncApp keys.

As I see it it could be implemented as a filter on SyncApp's filesystem access so that whenever it reads a file from the sync directory (and probably the directory listing itself) it encrypts the data and provides the encrypted data to SyncApp proper. It works kind of like an upside down encfs, with the data on the underlying filesystem in clear text and the virtual filesystem provided to SyncApp being encrypted.

This way any application that doesn't have the filesystem password will just see a normal filesystem with encrypted files (Which could be decrypted by some independent tool). A SynchApp with the password would put the files through the filter layer before they were written to the disk. Only the filter layer would see the cleartext data.

BTW: The large majority of filesystems don't have resource forks, also NTFS has alternate streams and encryption which can make themselves known in annoying ways. This technique, adding a filesystem filter can be used to sync resource forks and alternate streams to filesystems and OSs that don't need compatibility with these strange or exceptional features.

Conclusion: Perhaps it's already available as a library somewhere; but something needs to be done about symlinks and junctions.

Share this post


Link to post
Share on other sites

This would be moderately difficult to implement IMO, but it's not part of the 'sync' process at all.

I really don't know. Is the AES encryption on top of the data streams or is it on top of the data itself. It's quite a difference. Socket encryption or File (block) encryption. Only a developer could tell I guess.

@Kos13, is there any take within the Bittorrent development on this?

dry.png

Share this post


Link to post
Share on other sites

+1 for the encrypted & read-only secret.

This would allow for a nice multi-user setup, which is currently next to impossible to do.

@GreatMarko, your argument doesn't hold : File1.rar would be synced back on Client B as File1.txt, not File1.rar !

Share this post


Link to post
Share on other sites

+1 from me as well. There are plenty of use cases where this would be helpful. Most important in my mind would be replicating data to read-only untrusted locations where it can be downloaded and decrypted even in the event of a local data loss.

a quick question: what would happen if the read-only client didn't have the actual secret key but just the SHA2(Secret)? Could it make a request to receive encrypted data from the sync sources? Encrypted data which obviously it would not be able to decrypt.

Share this post


Link to post
Share on other sites

I really don't understand how this could be in the scope of this project.

Ye olde TrueCrypt or your newfangled EncFS or the distributed cloud services listed sound far more like what you're looking for.

Share this post


Link to post
Share on other sites

I just requested this yesterday. +1

EncFS, ecryptfs, truecrypt, whatever is not a viable solution because for them to work the storage needs to be mounted and that is an easy target if anybody has root access there. If you're thinking puting the encrypted storage in the sync that would work of course, but I'd only want storage level encryption at the remote, untrusted host.

Share this post


Link to post
Share on other sites

+1. Please reconsider adding this.

Using encrypted FS over shared data is not a solution because if we use it anyway, what are the benefits of BTSync? How is this different from EncFS over Dropbox or even FTP? (And we cannot sync only changed files which effectively destroys the idea of sync)

Encrypting target FS on unprotected PCs is pointless as someone who has access to the PC has access to its mounted filesystems.

And there are many use cases for this:

- I want to synchronize two computers (home PC + work PC) but I don't have both running at the same time. I set up a server on a hosting and run BTSync there, PC1 syncs with Server, Server syncs with PC2, everyone happy. Hacker hacks server, has access to all my data.

- I'm a businessman, I see this perspective new technology called BTSync and realize I can run my own node and provide hosting for case #1 people. I'm all ready to earn money (and donate to BTSync team) but people don't trust me with their data. If only I could handle it in encrypted form!

Share this post


Link to post
Share on other sites

I really don't understand how this could be in the scope of this project.

...

I disagree, I think the fact that the team is working very hard to make sure it runs on NAS devices makes this feature fit very well with the scope of the project. Having your backup data encrypted is very reasonable. Also using BT Sync with a cheap VPS is an amazingly disruptive private dropbox solution, but useless without file level encryption at the remote end. This is a feature no one else has.

Share this post


Link to post
Share on other sites

If you're looking to pass encrypted content to and from devices, BTSync is not what you're looking for. Try looking into FreeNAS, which offers Sync clients with the option to encrypt.

Share this post


Link to post
Share on other sites

It's not about mountable/usable encrypted storage. The use case would be seed servers and backup locations.

BTSync wouldn't have to decrypt the AES-256 encrypted transfered data, just keep it as is.

The problem with the implementation is the file indexing.

Share this post


Link to post
Share on other sites

I understand what people are getting at here, but I think using external encryption is the way to go. It makes it the most flexible, when you "bake in" certain features like disk encryption in folder sync tool, you inherently limit it's flexibility. Don't misunderstand me, I'd love to have a plugin for truecrypt or something like that, which triggers a sync after a data update or similar. Plug-ins are the way to go with something like this. There's no need to reinvent the wheels that truecrypt or 7zip have already invented.

Share this post


Link to post
Share on other sites

Please be aware that encryption is already in the BtSync product. Only on a network level. Since in my first post I talked about technically in-adept people, it WOULD be a nice to incorporate in into the simple secret exchange that BtSync now provides.

And furthermore, we are talking about file encryption, not disk encryption or container encryption here. Please bare in mind that these are truly different. For me, this still stands. It would provide me with a really great solution for helping my family.

Edited by PeterVerhees

Share this post


Link to post
Share on other sites

Huh? BTSync *already does* encryption, so they already reinvented the wheel (if you want to call it that). All they have to do is skip the decryption at selected nodes after receiving the file.

Nobody is talking about disk encryption.

Also, 7zip or Truecrypt didn't "invent" anything.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.