AcostaJA

Cloud Distribuited Sync With Partial Data Plus Parity.

Recommended Posts

This is an special request for an PHP version of BtSync (or other Web Hosting Farm (also NAS) Friendly Version), also for multiple NAS installs.

 

Since Web Hosting usually provides high availabilty, its plausible do not need to keep the full parts of a file on each host providing suport for the BtSync cloud (exept if said host will use sync'ed files internally).

 

Assume I have N hosts, I could distribuite the blocks of a single file among then, plus a parity block on each host, the parity its designed in case some host is deleted or unavailable it could be used to rebuild the missed part.

 

How I'll do that:

 

If I want to distribuite a File with N Blocks of data, distribuite it across the Active hosts, this way:

  1. Bk1 on H1, Bk2, on H2, Bk3 on H3 (only 3 -n- host)
    1. then save at each host a Parity Block Calculated from Bk1+Bk2+Bk3;
  2. repeat again the with rest of the block: Bk4 (n+1) on H1, BK5 (n+2) on H2, Bk6(n+3) on H3.
    1. save on each host the Parity Block of Bk4+Bk5+Bk6
  3. ....

This way if a file its requested from a diversity of host distribuited among N webhosting services, the amount of data stored at each host will be Size/N x (nParityBlock+1).

 

This scheme its storage efficient, and could be a Work Request in case some BtSync service its offered by WebHosting Farms.

 

My 2 cts

Share this post


Link to post
Share on other sites

A PHP version of BTSync seems a terrible idea to me.

Also most Web hosters kill PHP scripts that are running more than a few seconds.

 

Storage is cheap and will get even cheaper in the future - your "RAID" scheme seems a bit elaborate for me when I can just use one cheap Backupsy (or whatever) server as backup/distribution node and the original files on my PC.

Share this post


Link to post
Share on other sites

A PHP version of BTSync seems a terrible idea to me.

Also most Web hosters kill PHP scripts that are running more than a few seconds.

Storage is cheap and will get even cheaper in the future - your "RAID" scheme seems a bit elaborate for me when I can just use one cheap Backupsy (or whatever) server as backup/distribution node and the original files on my PC.

1. PHP scripts running in background are possible to keep alive thru N ways (name it tricks).

2. Is not about storage cost, but safety, if I put only a partial content of an encrypted file, it's impossible to rebuild it if the security is comprised (logically no file stored in a cloud serve maybe sent plain).

Share this post


Link to post
Share on other sites

1. "Tricks" is not what comes to my mind if the topic is safety.

 

2. That's a completely new area (you named availability as the motivation in your first post). If you don't trust the host, don't use it or encrypt the data. If the encryption is broken or compromised (which means someone has your secret) it doesn't really matter if the data is stored all on one or distributed across many hosts. 

Also, what's keeping the compromised host(s) from using your parity block to recover the rest of the data, provided they can't just download it from the other hosts?

Also, if the data is interesting enough, even leaking a part of a file (think videos, or word documents) might be enough to consider the attack successful.

 

Sorry, I see neither the benefit of PHP over native hosting nor the attack vector your proposal tries to protect against. Could you elaborate?

Share this post


Link to post
Share on other sites

1. "Tricks" is not what comes to my mind if the topic is safety.

Trick it's just a word, while the method used deliver stable operation it's plausible to deploy to production.

2. That's a completely new area (you named availability as the motivation in your first post). If you don't trust the host, don't use it or encrypt the data. If the encryption is broken or compromised (which means someone has your secret) it doesn't really matter if the data is stored all on one or distributed across many hosts.

Also, what's keeping the compromised host(s) from using your parity block to recover the rest of the data, provided they can't just download it from the other hosts?

Also, if the data is interesting enough, even leaking a part of a file (think videos, or word documents) might be enough to consider the attack successful.

Sorry, I see neither the benefit of PHP over native hosting nor the attack vector your proposal tries to protect against. Could you elaborate?

*from now I'll name Web based peers (hosts) as splitted peers, and non-splitted peers as std peers (conventional btSync peers) . *

1 as rule the host will only receive the partial data assigned, plus the parity, no host rebuilds data, only the std peers are capable to *assembly* the file, no splitted peer must be aware where are the other splitted peers, only contact the std peers, only std peers are aware the splitted peers location.

2 since It's required N-1 Blocks plus the parity block to rebuild the data, no peer will have N-1 blocks, only 1 block plus the parity block, if your clould comprises 3 splitted peers and a single host is seized or comprised, this host requires at least to download an extra block for another splitted peer, this thing will not happen since the rest of the splitted peers only can deliver its content to an std peer.

3. Compromising the network security is not a problem addressed by this feature, only the availability and security of stored data at SAN level on an remote host.

Share this post


Link to post
Share on other sites

Okay. So why do you think encrypting the data is not enough to protect against a malicious/compromised host?

 

And why can't a compromised host simply use the secret that it already has (be it R/O or R/W or Encrypted R/O) to get the rest of the data from the other split peers?

Share this post


Link to post
Share on other sites

1- when safety matters, never is enough.

2- split peers are passive with respect other splitted peers, don't know their ip address, location etc, to rebuild a file are required n-1 parts plus the parity, splitted peers only have one single part and the parity

Share this post


Link to post
Share on other sites

1- when safety matters, never is enough.

Okay, I'll stop discussing here. That's a killer phrase, not an argument.

 

split peers are passive with respect other splitted peers, don't know their ip address, location etc, to rebuild a file are required n-1 parts plus the parity, splitted peers only have one single part and the parity

That's naive thinking. If I take over a host with your PHP script running on it, I have your sync secret and can just use that to play client instead of split peer.

Share this post


Link to post
Share on other sites

That's naive thinking. If I take over a host with your PHP script running on it, I have your sync secret and can just use that to play client instead of split peer.

That's relative, you think on deploy the same btSync on the splitted peers, I think on an specific btSync for splitted peers with the capability to send parts to other splitted peers disabled totally even w/o dht info which allows it to find nothing than other std peers.

Naive thinking is what I name when someone don't provide ideas instead critique recursively with weak reasoning or acting like dumb, sorry actually this is not naive thinking, this is trolling.

Thanks for your time, my hope is that the idea behind this feature request will be well understood by btSync team.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.