Fail with large amount of data accidentally deleted and moved to archive on remote encrypted and local normal nodes

[carl566]

Ok, so good news is no data has actually been lost. But there has been a slight issue and trying to decide the best way to fix it 😐

Was decommissioning a remote UnRaid / Resilio Sync node which is at a friends house the other side of the world, all the data had been sync'd across his local LAN to a new Synology NAS with a new Resilio Sync node. The new NAS had the data encrypted by Resilio Sync.

I then disconnected the old Resilio Sync node from my identity, it looked like it had done it but unknown to me it hadn't.

I then started deleting out the data from the old server being decommissioned, but it was unknown to me still linked and Resilio Sync and proceeded to think all the data (a lot) was being deleted and moved it to the Archive folders on both the encrypted NAS node and my local node(s) on my laptop etc here local to me.

So I now have all this data in Archive, across every node, the problem is if I was to start fresh it would have to transfer TB+ of data back to the remote node.

I did a test and tried copying some of the encrypted data that had been moved into Archive back into the main folder on the remote NAS encrypted node, but it instantly moved it back into Archive so didn't recognise it as data it wanted to index.

Does this mean there is no other option to allow all this data to be re-encrypted and copied over WAN?

I did think of stopping Resilio Sync on the remote encrypted node, then on one of the un-encrypted local nodes moving the data back from Archive and letting it index itself, then on the remote stopped encrypted node moving all the encrypted data from Archive back into the normal share and then starting that node again and seeing if it would index/recognise the encrypted data. But I don't hold much hope of that.

FYI there has been no changes to any of the data anywhere, so in theory the encrypted data (I think) should be identical to what would be re-encrypted and synced over fresh.

Anyone have any other ideas? Like I say no loss of data which is great, but just seeing if I can get things back correct again without having to wait weeks and sucking up bandwidth to resync fresh.

Thanks!

[Helen]

unfortunately  you cannot restore the files from Archive on an encrypted node. Encrypted node - is a read-only node. Since the files are "deleted" on the source peer, the encrypted node cannot change this.
See other peculiarities about encrypted folders here https://help.resilio.com/hc/en-us/articles/207370466-Encrypted-folders 

Disconnecting form an identity only removes Advanced folders from sync. Standard folders remain and continue syncing. Encrypted folder is a standard folder. Therefore it propagated deletion as peers were still connected to each other. To be on safe side, manually remove the shares from Sync before doing something bad with the files.

[carl566]

17 hours ago, Helen said:

unfortunately  you cannot restore the files from Archive on an encrypted node. Encrypted node - is a read-only node. Since the files are "deleted" on the source peer, the encrypted node cannot change this.
See other peculiarities about encrypted folders here https://help.resilio.com/hc/en-us/articles/207370466-Encrypted-folders 

Disconnecting form an identity only removes Advanced folders from sync. Standard folders remain and continue syncing. Encrypted folder is a standard folder. Therefore it propagated deletion as peers were still connected to each other. To be on safe side, manually remove the shares from Sync before doing something bad with the files.

Thanks, that was interesting.

It does say "To get the deleted file back either restore them from recycle bin, or from Archive on other connected Read-Write peers. "

Which I can do, now if I do that would the Encrypted read only node realise it already has these files encrypted in the Archive and copy those back (eliminating the need to download them again) or would it just download them again?

I will look again this evening and try and resolve the issue one way or another  Thanks

[Helen]

it shall move out of the archive provided the hashes of files that you will be restoring and of files in the archive are still matching, -  if the files didn't change which actually can happen. For example, if files timestamps changed after you deleted them and restored , that would be considered a changed file. 

Try on a bunch of files first to see. . You might track it in logs on encrypted node, search for lines like "Found file with hash blah-blah in trash..." or alike, sorry, don't remember by heart. 

[carl566]

5 hours ago, Helen said:

it shall move out of the archive provided the hashes of files that you will be restoring and of files in the archive are still matching, -  if the files didn't change which actually can happen. For example, if files timestamps changed after you deleted them and restored , that would be considered a changed file. 

Try on a bunch of files first to see. . You might track it in logs on encrypted node, search for lines like "Found file with hash blah-blah in trash..." or alike, sorry, don't remember by heart. 

Well this is great, yes I cut/paste (so moved not copied) the files on my local RW node here from the Archive back into one of my shares to test. It started indexing locally, and almost instantly the files started showing on the remote encrypted node. I could see there was no significant network traffic and within seconds several GB of files were starting to show up.

So I don't think I even need to check the logs, I'm certain it's finding the hash of the files and brining them back out of Archive on the remote encrypted node without having to resend the files.

It makes sense as like I'd said I made sure not to change any files and hold fire to see what was best.

Thanks for your help, and hopefully this thread helps someone else in the future