Recommended Posts

I am trying to sync LARGE folder from a Windows Machine to any other device (at the moment a Raspberry Pi with a USB Drive, but if I get this working it will be any other device).  Large is just shy of 3TB.  My thinking was that while I have access to USB I could pre-sync the folder by simply copying files over and then running BTSync to mirror any additional changes.  Sync of course wanted to overwrite every file on the Pi.  I am assuming due to changes in the file attributes triggering sync to replace them.  Trying to use Sync across the lan will take roughly a Month and is outside the bounds of my timeframe.  My next try was to use a Robocopy /copyall which in theory should make an exact duplicate of the file including attributes (created/modified timestamps etc).  Still wants to overwrite the entire contents of the drive.  I am a little stumped, and hopefully there is a way to achieve this?  I really like Sync and dont want to have to resort to rsync and chron to automate mirroring my drive.  

 

tl:dr

How do you manually copy files from a source directory to a target directory and then have Sync maintain them as mirrored without overwriting every file in the target.

 

PS: Apologies if this is a trivial question.  I'm new to sync, and spent most of yesterday on Google, and these Forums trying to figure this out.  

Share this post


Link to post
Share on other sites

Sync of course wanted to overwrite every file on the Pi.

Still wants to overwrite the entire contents of the drive.

You're assuming this based on what behavior exactly?

It still has to index the files on the pi even if you copied them over from an existing source. Pre-copied files will be indexed, their index information compared, and only replaced if the index information (which is based on segmented checksums of the files) differs.

Share this post


Link to post
Share on other sites

In all attempts I mentioned after setting it up, the Pi would eventually say "Out of Sync" and the hover text would say 2888.34 GB of updates. Neither Source nor Target indicated indexing in any significant way, although again I am new, not sure how to tell for sure.  At one point it said "Indexing" under status I believe and now it says out of sync.  Am I wrong to assume that means the indexing is complete?


Starting the sharing process from scratch so I can pay a little more attention and provide better info.  

 

Added a new Shared Folder:  Status = No Peers - indexing

 

While still Indexing got a read only shared link, and added to the Pi's bittorrent webgui.

 

-Destination not Empty.  Add Anyway. - Yes Approved the request.

 

Green Check - Indexing on the Source

Green Check - no text on the Target

 

Both now say Out of Sync with an Up Arrow on Source

When I click the peers link on Source it shows the Target machine with an Up Arrow and a growing count of Gigs/Files.  That implies that it is already finding differences across the files it has indexed, does it not?  This is consistent with what I saw before, and eventually it gets to a state where it wants to upload a FileCount and Size that is consistent to the sum of my files.  

Edited by Triumph23

Share this post


Link to post
Share on other sites

The gigs of difference also includes unindexed files on the destination.

As the destination indexes and as the source polls the destination, the difference amount will go down without transferring the files.

Share this post


Link to post
Share on other sites

Indexing completed.  Both systems show 1 of 1 Peer, but out of sync with the full load of files to be transferred, but not transferring anything.  Actually, when I checked both peers show each other as offline.  They are both certainly online (I'm looking at both of them at the same time on the same Lan, can ping between them etc).

 

Well shoot, this is a whole new problem now...

 

But still, it seems as though they have communicated enough to know they are out of sync, and both are showing needing to send or receive the entirety of my content.

Share this post


Link to post
Share on other sites

@Triumph23

It simply indicate both peers did not manage to merge the tree of files. Could you please:

1. Enable debug logging.

2. Extend the log_size advanced preference to around 500Mb?

3. Provide logs for analysis.

Share this post


Link to post
Share on other sites

I second this request. I would love the ability to manually copy the latest sync folder from one computer to another before initiating sync on the latter.

 

I work in a small design and branding shop and we have been syncing our work files for a month now. In that time, we've had to setup and begin syncing 4 separate computers. Each time a new computer is added to the peer list, we end up having to wait so long for the sync to complete. It would be so much more convenient to transfer the sync folder to an external hard drive, transfer it to the new computer, and then initiate sync on it.

Share this post


Link to post
Share on other sites

@captainpease

This functionality is already implemented - try it. When adding folder that already has pre-synced data, Sync should just index it and do not transfer files (directory structure should be completely same).

Share this post


Link to post
Share on other sites

On this topic is it safe to reconnect a disconnected sync folder?

 

Yes - any changes made to the contents of the folder whilst it was disconnected (or the other folders it syncs with) will then be transferred once it is reconnected

Share this post


Link to post
Share on other sites

@captainpease

This functionality is already implemented - try it. When adding folder that already has pre-synced data, Sync should just index it and do not transfer files (directory structure should be completely same).

I am trying to do the same thing, sync the Documents folder on my desktop, to the Documents folder on my laptop. I have tried doing it in both directions, but what happens in both cases is that it creates a duplicate Documents folder under Documents. My desktop is Windows 7 and my laptop Win 8, is the problem with the directory structure?

Share this post


Link to post
Share on other sites

@Michael Horton

Could you please let me know exact path you try to add? Many user folders in Windows are actually not folders but Directory Junctions. Sync does not work well with Junctions. So right solution here would be to add real path Junction points to.

Share this post


Link to post
Share on other sites

Indexing completed.  Both systems show 1 of 1 Peer, but out of sync with the full load of files to be transferred, but not transferring anything.  Actually, when I checked both peers show each other as offline.  They are both certainly online (I'm looking at both of them at the same time on the same Lan, can ping between them etc).

 

Well shoot, this is a whole new problem now...

 

But still, it seems as though they have communicated enough to know they are out of sync, and both are showing needing to send or receive the entirety of my content.

 

@Triumph23

Did you ever figure out why it was trying to sync all the data even though the folders were identical?

I'm having the same issue with v1.4.111. Apparently it's going to take 5 months to sync even though the directories are identical.

Share this post


Link to post
Share on other sites

Unfortunately no.  I ended up bailing on using Sync to get this done, and went with an FTP syncing script that runs @reboot in crontab.  Less elegant but it gets the job done.  Unfortunately my system is deployed and I cant get to it for another couple of months.  Once I do, I am looking forward to trying Sync 2.0.  This project is such a cool idea, hopefully the kinks will get ironed out soon.  At the time of writing my first question I believe Sync 2.0 was in alpha and unavailable for a Pi (without a headache).  Haven't researched thoroughly yet, but I am HOPING there is a simple repo install, or at least an easily downloadable package to install 2.0 on a Raspbian machine.

Share this post


Link to post
Share on other sites

Is this working within Sync 2.0?

Copying a beta version of the folders content to the second peer, add it as read-only, and it transferrs the new files and delete the ones i deleted on the owner-peer?

Share this post


Link to post
Share on other sites

@12345lamacun

Yes, it does. Although, if some files are newer on your owner-peer they won't be replaced unless you ensure the checkbox "Overwrite changed files" is checked in folder properties of RO peer.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.