PeterVerhees

[Now Implemented!] Support For Untrusted Encrypted Node.

Recommended Posts

You read my earlier message; the one that says what you need is an upside down encfs ... well, seems it exists ...

--reverse

Normally EncFS provides a plaintext view of data on demand. Nor‐

mally it stores enciphered data and displays plaintext data. With

--reverse it takes as source plaintext data and produces enciphered

data on-demand. This can be useful for creating remote encrypted

backups, where you do not wish to keep the local files unencrypted.

For example, the following would create an encrypted view in

/tmp/crypt-view.

encfs --reverse /home/me /tmp/crypt-view

You could then copy the /tmp/crypt-view directory in order to have

a copy of the encrypted data. You must also keep a copy of the

file /home/me/.encfs5 which contains the filesystem information.

Together, the two can be used to reproduce the unencrypted data:

ENCFS5_CONFIG=/home/me/.encfs5 encfs /tmp/crypt-view /tmp/plain-view

Problem solved :)

Share this post


Link to post
Share on other sites

@rdebath: Yes I've read your earlier messages. I even mentioned EncFS myself in my initial posts. But I said, and I quote: "technically in-adept people".

EncFS is all fine and I use it daily on several *nix machines and even a Windozebox for cloudsyncing to the NSA/FBI/CIA PRISM storage facility and other cloud providers..., however I am *NOT* technically in-adept.

Share this post


Link to post
Share on other sites

Problem solved :)

Really?

Please show us a way to wire it together with BTSync without the risk of exposing the key of the original data. :)

Should I create a virtualized system with an OS that even encrypts the RAM?

Implementing it in the client is a good idea indeed, but keep in mind that this shuts out Windows [8] as a host.

I could settle for a Unix-only encrypted nodes feature nevertheless.

Share this post


Link to post
Share on other sites

> Problem solved

As I see it, there are only two ways to go about external encryption here:

1. Encrypt data BEFORE you sync it to untrusted PC.

2. (Re-)Encrypt it AFTER it has been delivered to untrusted PC.

In the second case, whatever cool encryption you use, it's not real security because the data is decrypted at some point on an unprotected PC. And BTSync must be able to access so the key is also available unprotected. Right?

In the first case you lose all the benefits of sync. "File-grained encryption, re-upload only changed files", if one file changes you have to re-upload the whole share, if one file changes you have to reupload the whole share. If someone else changed one other file, both shares conflict. Right?

Share this post


Link to post
Share on other sites

if one file changes you have to re-upload the whole share

Why? You could still sync encrypted files separately, not one single huge blob.

Edit: Yes, if you used a container-like external encryption (which I wouldn't want), you would have a huge blob. But changing a single file in the container shouldn't change the whole container, only the chunks where the file sits.

Share this post


Link to post
Share on other sites

I'd like to jump in with support for this feature. I'm currently using duplicity (http://duplicity.nongnu.org/) for this functionality, but it would be great to switch over to the more user-friendly btsync. I don't know (or understand) the details, but duplicity seems to have a way to do incremental transfers securely on encrypted data. Maybe their method is something that could be adopted by btsync?

Share this post


Link to post
Share on other sites

ChrisH: Maybe you're right. Anyway, it's not simple. You need some file-level encryption, different keys for it, and make BTSync see encrypted data while the rest of the system sees decrypted one. And that supported on every device/OS BTSync runs on (not just the storage one).

Share this post


Link to post
Share on other sites

I would love to hear the opinion of a dev.

How likely is it that we see such a feature in the near future? Next release, next half year, somewhen in 56 years or is it simply impossible?

Share this post


Link to post
Share on other sites

Hey there.

A someone already wrote: I go for EncFS, which works pretty well.

  • The path "/var/storage/encrypted" contains all encrypted data, which is plain file system stuff.
  • The path "/var/storage/encrypted/ljwsvhuasflkh1" could be the encrypted path name for my "music". Pain file system stuff here, too.
  • The path "/var/storage/encrypted/quihsgdnhuiaqef3" could be the encrypted path name for my "documents", and plain file system stuff once again.
  • EncFS mounts and decrypts "/var/storage/encrypted" to "/var/storage/decrypted", through EncFS it's kind of virutally
  • That means "/var/storage/decrypted/music" is the decrypted version of "/var/storage/encrypted/ljwsvhuasflkh1"
  • And that means "/var/storage/decrypted/private" is the decrypted version of "/var/storage/encrypted/quihsgdnhuiaqef3"
  • Now btsync kicks in. It uses "/var/storage/encrypted/ljwsvhuasflkh1" as my first sync/share
  • And it uses "/var/storage/encrypted/quihsgdnhuiaqef3" as my second share

The advantages of this setup are:

  1. If my device gets lost (say, stolen), my data is as secure as a lost device could be. Whoever finds it can access encrypted data only. He might have access to my sync secrets, but that doesn't give him access to unencrypted data.
  2. I use only a single encrypting base ("/var/storage/encrypted") with a single password. But I have different syncs in there, which allows e.g. my personal computers to use all of my syns, where my office computer only syncs the "private" share.
  3. I can setup backup/seeding nodes just by renting a cheap and semi-thrusted vps somewhere. As long as I only set the btsync config without adding the EncFS config, the backup/seeding node does not have access to unencrypted data.
  4. The encryption configuration file (I think it's called encfs.xml or something) is not part of the synchronized data. To setup the encfs part to a new node, I don't have to simply type in the password in an EncFS prompt but I need to manually copy the configuration file first. That's kind of even more security ontop, since the encrypted data isn't "fully self-contained".
  5. EncFS is available for Linux, Unix, MacOS, Windows and Android. I don't know if there is something for iOS. I don't think so, since iOs doesn't provide file system level to applications. So this setup might not work with iOS. But that's not a blocker for me.
  6. The encrpyted data contains as much files as the unencrypted folder has, and each encrypted file has an equivalent decrypted file with almost the same size (adding a view bytes of metadata, but not worth talking about it). This results in very small files to be transferred and no (noticable) block shift problem.
  7. I don't habve to rely on btsyncs closed source encryption feature only. No offense to the owners, it's a pretty nice tool. But with btsync having only access to encrypted data created by open source EncFS makes me sleep even better. In the early days of btsync, I thought about "man, would be pretty nice if btsync would be able to make transparent EncFS internally". But as of my current state of knowledge and worldview, I like keeping file system level encryption and device synchronization in different products more than having a single closed source product handling all my personal data.

The only dissadvantage I see here is: Plain btsync dosn't see unencrypted file names, so on-demand syncing on android devices is kind of impossible. You never now if /var/storage/encrypted/sdfljwregs/sdolkgw3lhfsd/sdlfknwef/sdfsdfsd is the song you want to listen to or just the android ".nomedia" file indicating that this particular folder should not be added to androids media index.

But that's O.K. for me, I simply put all my syncs to "automatic synchronization".

There are a view little API things that would improve my btsync setup, such as "execute a command line tool as soon as the first remote finishes syncing". But that's absence defintively doesn't make it unusable.

Regards,

Stephan.

Share this post


Link to post
Share on other sites

You also have to give up SyncIgnore unless you use the encrypted paths in it, have fun with that.. Not to mention getting EncFS to work on Windows 8 is a hackathon. You also loose a lot of I/O performance with file level encryption. I say go with disk encryption and sync decrypted folders. If you want untrusted seeding nodes, you're gonna have to wait until there is such feature in BTSync natively or over an API.

Share this post


Link to post
Share on other sites

I agree that this would be an important feature. It would allow semi-private environments.

For example someone who is tech savy and owns a dedicated server could offer bandwith (for caching) and storage to his friends and family without gaining access to their files.

Without knowing the exact implementation and the protocol I guess it at least should be able to add a third type of secret in addition to "full access" and "read only" which would be "caching only".

This key would allow the local sync client to download and cache files in encrypted state. This adds two important features:

  • I still have access to my data in case all other of my sync clients are turned off
  • I can benefit from fast download speeds of the caching server (most residential lines are only async)
  • The person providing the cache server would not have access to the data that his server caches

In fact this would allow a whole new business for service providers to provide caching of sync data against a small payment while still having no access at all (even if they'd try) to the data.

Share this post


Link to post
Share on other sites

+1 from me too. While I admit this is not so urgent or must have feature, I do wish it were natively implemented into the client. I already use read-only secrets to have additional seeds, caches and backups. And I know very well, that many of those cases be better unreadable for those middlemen, not only unwriteable. I would totally use such feature myself and gladly provide my broadband and storage to provide seeds for my relatives too.

Share this post


Link to post
Share on other sites

But is there a stable implementation of encfs for windows?? encfs4win is experimental, as stated on their website.

So I still would love to know, will it be possible to implement this feature out of the box to BTSync? :D

Share this post


Link to post
Share on other sites

Another +1 for this feature request.

I have implemented the encfs + btsync solution on my server. I can now securely sync data between my laptop (running linux) and my phone (Android using Cryptonite). However, it requires quite a bit of tinkering and tech savviness. An integrated solution would make this more accessible to the average user.

My current setup is not ideal on my phone... as I have to sync all the data before I can access it through Cryptonite. For Dropbox there is a special mode in Cryptonite which allows one to access data without sync'ing it to the phone... you select a particular file, which is then decrypted and exported/opened.

Please consider adding the "Read-only + Encrypted mode" in btsync.

Share this post


Link to post
Share on other sites

+1 for the "read-only and encrypted mode" feature from me.

I'm requesting this as a windows user with encfs4win having issues with some file types (i.e. MS Office).

Is there another free "transparent folder-level" encryption software for Windows?

Share this post


Link to post
Share on other sites

@several posts in this thread.........

Can we please stop plugging other solutions like EncFS/Truecrypt/Lukscrypt etc etc. People interested in this feature most likely have already checked those out. I myself mentioned EncFS in one of my first posts just to say I am aware of it, no more.... It keeps coming back as a solution. It's not a solution for many people and as mentioned in posts before me, not even stable on Windows.

I opened this post as a feature request. Can anyone from the staff give us an answer as to whether or not this is even on the radar of the developers?

As for myself, this would be an option I would gladly pay for.

Share this post


Link to post
Share on other sites

Hey there.

Don't get me wrong. I do see the request for embedded encryption. But that's not a point to avoid and disqualify EncFS completely.

I tried to combine both here in the wishlist thread:

Pretty much as you, Peter, I requested encrypted-only secets ... or morely I supported another guy requesting this.

Ontop of plain "please give us encrypted nodes" I tried to request and explain: It would be pretty nice, if btsync supported EncFS compatible encryption; If btsync would behave according to EncFS protocol when doing encrypted-only synchronization.

I don't know if it's this thread or another, but somebody already explained that EncFS has basically two modes of operation.

  • The first one uses encryted files on a file system level and presents the unencrypted content virtually as a mount
  • The second one does it the other way around. Files are unencrypted on your file system but presented virtually as an encrypted mount.

The second mode is obviousely much faster than the first one because usual file access doesn't go through encryption. Only the syncing part would access the encrypted vrtual mount.

And here is where my suggestion kicks in: Maybe it's possible to have the btsync app use the EncFS library to do on-the-fly encryption of data right before the data is sent out.

On my local machine, btsync behaves like it does currently.

We add a "emcrypted-only secret".

If a node connects with a "encrypted-only" secret, my local computer does EncFS protocol like encryption of file contents before sending it to the remote host.

Done.

This would allow for a couple of things:

  • It's kind of reviewable, because the EncFS protocol is open. So there's no problem to check weather btsync does encryption in a secure way or not.
  • If there are platforms where EncFS is not available, not part of btsync or maybe even if there is any situation where no btsync at all is available when accessing a previousely synced folder (I'm thinking about access to backups of encrypt-only folders), the unencrypted stuff can still be recovered.
  • There's not so much performance overhead during regular heavy file access compared to unterlying file system level based encryption.
  • This would allow for a paranoid start and migration to less paranoid. I could start with EncFS as a virtual file system leve encryption and put btsnc ontop of it. Just like explained a couple of posts ago. And in a couple of month, as soon as I discovere that EncFS on windows isn't as stable as I thought, I could just press the "migrate to less paranoid EncFS" button, import the EncFS configuration file into btsync on my local computer, and from here on all other nodes do act as "encrypted-only" nodes without the need to retransmit hundrets of GB of files again to my other nodes.

Especially the performance overhead thing is a point, imho. Think about virtual machine hdd files. VMDK for VMware, if you want. Those keep the file system bussy constantly.

If there's file system level encryption (If I drop my VMs into an EncFS folder), constant encryption is necessary. I have VMs that keep my HDD 60MB/s for several hours.

If the encryption is not on a file systsem level but a part of btsync then there is no need to encrypt data as often as possible but only when synchronization is going on.

Therefore the btsync is required to have consistent index data for both, the encrypted and unencrypted version of a file.

So: I do see the need for an integrated encryption.

But nevertheless I do want to have it based on a commonly known standard as well. And I suggest EncFS because of its advantages for single file encryption and its availablility as a implementation as compiled ready to use application but not only as a C function.

Ths is basically what I tried to request with my wishlist post.

And of course, that's something I would play a couple of beers, too. I don't know how much. To go for 100€ or more per license It would *require* to be transparent, imho.

Kind regards,

Stephan.

Share this post


Link to post
Share on other sites

@goli: There's already one encryption used when transferring data. To my mind it'd be better if a node could just "store encrypted blocks" it receives. I don't know if it's possible but it would be cool.

Share this post


Link to post
Share on other sites

Hi himselv.

I'm aware of the fact that btsync uses transport encryption. I didn't dig into this, which would be very difficult due to the closed source stuff. But I really hope that they do not only encrypt paiload file contents. I hope it is kind of an unencrypted protocol for transmission and having commonly known TLS wrapped around. That's transparent, easy to handle, proofed to be secure and used in that exact way everywhere all over the place. And if it's done the way I hope, then there simply is no encrypted file content block that can be stored, cached and reused. Having TLS wrapping a plain unencrypted protocol means skipping the encryption completely destroys delta syncs or comparing indexes and file hashes. It completely destroys the whole sync mechanism.

This post should not be against any kind of block encryption and encrypt-only secrets. It's just an explanation why I think it's not an easy task to simply "skip the encryption function on the encrypt-only nodes". I really do want encrypt-only nodes. Having nodes that only hold, say, 4MB chunks and an encrypted indexing table that don't allow for any kind of reconstruction or file guessing would be awesom. But i think that's completely different than the current mechanism and therefore really unlikely to be implemented.

Regards,

Stephan.

Share this post


Link to post
Share on other sites

@goli: I hope for the opposite. If they're using just TLS, then everything needs to be done from the scratch; on the other hand if it's SpiderOak-like block-level encryption then delta syncs are already implemented. And since this is BitTorrent... do torrents really use TLS? How would it work, if my data is rooted through several other nodes?

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.