atomicbird

Issues moving sync to a new directory

Recommended Posts

A few days ago I needed to move a shared directory to a new location, since the old location was running out of space. This is on Ubuntu 16.04, and it's using an encrypted key. I initially set up Resilio using the steps described at https://help.getsync.com/hc/en-us/articles/206178924-Installing-Sync-package-on-Linux

To move the store I did this:

  1. Used the Linux web UI to disconnect and remove the existing location
  2. Copied the contents of the share directory using "sudo cp -a [old location] [new location]"
  3. Copied the encrypted key from Resilio on my Mac, and entered it in the Linux web UI
  4. Selected the directory where I had copied the files back in step 2.

Now there are a couple of things that have me concerned.

  • The "indexing" step seems to be taking an extremely long time. On my Mac Resilio reports the size as 12.95GB. In the Linux web UI it's gradually growing. But, a little over 5 hours ago it was at 3.53GB and now it's at 4.03GB. At this rate indexing will take several days. Is this normal? The system is not maxed out for CPU, disk access, memory, or bandwidth, so what's holding it back?
  • My Mac reports that the total data size is 14.51GB, but on the Linux box the total file size (from "du -sh") is currently at about 26GB and growing. It's kind of a race to see if the indexing finishes before the partition runs out of space, which is odd since it's already using nearly 2x the space I'd expect.

Can anyone offer suggestions or advice on this? Did I screw something up, and if so what should I have done differently?

Share this post


Link to post
Share on other sites

To update this, shortly after posting, the Linux share started uploading files from my Macs instead of simply reindexing its own files. I had thought that my steps to move to a new directory would do the job without needing to re-upload most of the data, but it looks like I was mistaken. The partition I'm using doesn't have enough space to hold the >10GB of files still not accounted for.

This is looking like a disastrous failure. Where did I go wrong?

 

Share this post


Link to post
Share on other sites

As of just now, Resilio ran out of space on its partition. It's a 30GB partition, and Resilio on my Mac reports that this shared folder contains 12.95GB of data. I would have thought that setting aside >2x the expected requirement would be more than enough, but apparently not. This partition on the Linux box doesn't have anything else on it, only Resilio sync data.

Can anyone tell me what I should have done to move my working shared directory to a new location? I assume that I screwed up somewhere but I don't know where.

 

Share this post


Link to post
Share on other sites

I sort of got through this, but in the absence of better advice, I have to say that if you need to move a shared directory to a new location, just don't. You'll be better off creating a new copy and letting it sync.

In the case of my problem sync, described extensively above: I expanded the dedicated partition to 40GB. Eventually it completed. However, it's using 36GB of space on the partition for a shared directory that the web UI reports as containing 12.95GB of data.

Since that seems ludicrous, I set up another Linux VPS with Resilio and added the encrypted key there. It has now finished syncing, and is using 14GB of space for the same shared directory, which its web UI also reports as having 12.95GB of data.

I don't know what the original Resilio install is doing with that extra 22GB of space. Since it's a VPS I'm just going to destroy the instance and forget about it. If I ever need to move a shared folder in the future, I'll set up a new VPS instance and let it sync rather than try to move the existing directory.

However, if anyone knows the correct way to move a directory, or has any theories on why my move resulted in needing more than 150% extra disk space for the same shared directory, I'd love to hear about it.

 

Share this post


Link to post
Share on other sites

This problem appeared because you've  used a non-empty directly (already having lots of Gigs of data) to connect through an F-key on linux, right? 
Encrypted folder is a Read-Only folder. So it was ignoring all the files that were already there. Instead it was re-downloading the files, putting the current ones to Archive. I suppose you saw a warning "Destination folder is not empty, files will be overwritten", right?
This is why space taken grew and du showed that big size. 
Indexing will take long because the files are being re-downloaded and encrypted during transfer. 

Don't use a non-empty directory with encrypted share.
to move encrypted share to a new partition, re-sync the files. On Linux Sync does not follow share movements at all, On Windows/OS X - only within one drive. SO on latter one can move the whole share around, not the files themselves. 

Share this post


Link to post
Share on other sites

I don't recall if I saw those warnings.

Since there's no documentation on moving to a new directory, I guessed (incorrectly) that entering the same key would mean that Resilio could re-scan the contents. I didn't expect Resilio on Linux to automatically follow the share, I was trying to tell Resilio how to find it.

Assuming that I did see those messages, I'm not sure why Resilio left the existing files in place if the message warns that they'll be overwritten.

It would be useful if this were explained somewhere, so as to help people from making the same mistake in the future.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.