Triumph23 Posted November 14, 2014 Report Share Posted November 14, 2014 I am trying to sync LARGE folder from a Windows Machine to any other device (at the moment a Raspberry Pi with a USB Drive, but if I get this working it will be any other device). Large is just shy of 3TB. My thinking was that while I have access to USB I could pre-sync the folder by simply copying files over and then running BTSync to mirror any additional changes. Sync of course wanted to overwrite every file on the Pi. I am assuming due to changes in the file attributes triggering sync to replace them. Trying to use Sync across the lan will take roughly a Month and is outside the bounds of my timeframe. My next try was to use a Robocopy /copyall which in theory should make an exact duplicate of the file including attributes (created/modified timestamps etc). Still wants to overwrite the entire contents of the drive. I am a little stumped, and hopefully there is a way to achieve this? I really like Sync and dont want to have to resort to rsync and chron to automate mirroring my drive. tl:drHow do you manually copy files from a source directory to a target directory and then have Sync maintain them as mirrored without overwriting every file in the target. PS: Apologies if this is a trivial question. I'm new to sync, and spent most of yesterday on Google, and these Forums trying to figure this out. Quote Link to comment Share on other sites More sharing options...
Harold Feit Posted November 14, 2014 Report Share Posted November 14, 2014 Sync of course wanted to overwrite every file on the Pi.Still wants to overwrite the entire contents of the drive.You're assuming this based on what behavior exactly? It still has to index the files on the pi even if you copied them over from an existing source. Pre-copied files will be indexed, their index information compared, and only replaced if the index information (which is based on segmented checksums of the files) differs. Quote Link to comment Share on other sites More sharing options...
Triumph23 Posted November 14, 2014 Author Report Share Posted November 14, 2014 (edited) In all attempts I mentioned after setting it up, the Pi would eventually say "Out of Sync" and the hover text would say 2888.34 GB of updates. Neither Source nor Target indicated indexing in any significant way, although again I am new, not sure how to tell for sure. At one point it said "Indexing" under status I believe and now it says out of sync. Am I wrong to assume that means the indexing is complete?Starting the sharing process from scratch so I can pay a little more attention and provide better info. Added a new Shared Folder: Status = No Peers - indexing While still Indexing got a read only shared link, and added to the Pi's bittorrent webgui. -Destination not Empty. Add Anyway. - Yes Approved the request. Green Check - Indexing on the SourceGreen Check - no text on the Target Both now say Out of Sync with an Up Arrow on SourceWhen I click the peers link on Source it shows the Target machine with an Up Arrow and a growing count of Gigs/Files. That implies that it is already finding differences across the files it has indexed, does it not? This is consistent with what I saw before, and eventually it gets to a state where it wants to upload a FileCount and Size that is consistent to the sum of my files. Edited November 14, 2014 by Triumph23 Quote Link to comment Share on other sites More sharing options...
Harold Feit Posted November 14, 2014 Report Share Posted November 14, 2014 The gigs of difference also includes unindexed files on the destination. As the destination indexes and as the source polls the destination, the difference amount will go down without transferring the files. Quote Link to comment Share on other sites More sharing options...
Triumph23 Posted November 14, 2014 Author Report Share Posted November 14, 2014 I see. I'll give it some time to index, and come back with results. Thanks for the assist! Quote Link to comment Share on other sites More sharing options...
Triumph23 Posted November 15, 2014 Author Report Share Posted November 15, 2014 Indexing completed. Both systems show 1 of 1 Peer, but out of sync with the full load of files to be transferred, but not transferring anything. Actually, when I checked both peers show each other as offline. They are both certainly online (I'm looking at both of them at the same time on the same Lan, can ping between them etc). Well shoot, this is a whole new problem now... But still, it seems as though they have communicated enough to know they are out of sync, and both are showing needing to send or receive the entirety of my content. Quote Link to comment Share on other sites More sharing options...
Swingtrade Posted November 15, 2014 Report Share Posted November 15, 2014 Same thing here, it's not indexing and just going out of sync. No ways i'm downloading all that stuff again... Quote Link to comment Share on other sites More sharing options...
RomanZ Posted November 20, 2014 Report Share Posted November 20, 2014 @Triumph23It simply indicate both peers did not manage to merge the tree of files. Could you please:1. Enable debug logging.2. Extend the log_size advanced preference to around 500Mb?3. Provide logs for analysis. Quote Link to comment Share on other sites More sharing options...
captainpease Posted December 17, 2014 Report Share Posted December 17, 2014 I second this request. I would love the ability to manually copy the latest sync folder from one computer to another before initiating sync on the latter. I work in a small design and branding shop and we have been syncing our work files for a month now. In that time, we've had to setup and begin syncing 4 separate computers. Each time a new computer is added to the peer list, we end up having to wait so long for the sync to complete. It would be so much more convenient to transfer the sync folder to an external hard drive, transfer it to the new computer, and then initiate sync on it. Quote Link to comment Share on other sites More sharing options...
RomanZ Posted December 18, 2014 Report Share Posted December 18, 2014 @captainpeaseThis functionality is already implemented - try it. When adding folder that already has pre-synced data, Sync should just index it and do not transfer files (directory structure should be completely same). Quote Link to comment Share on other sites More sharing options...
jmr01 Posted December 18, 2014 Report Share Posted December 18, 2014 On this topic is it safe to reconnect a disconnected sync folder? Quote Link to comment Share on other sites More sharing options...
GreatMarko Posted December 18, 2014 Report Share Posted December 18, 2014 On this topic is it safe to reconnect a disconnected sync folder? Yes - any changes made to the contents of the folder whilst it was disconnected (or the other folders it syncs with) will then be transferred once it is reconnected Quote Link to comment Share on other sites More sharing options...
Michael Horton Posted December 19, 2014 Report Share Posted December 19, 2014 @captainpeaseThis functionality is already implemented - try it. When adding folder that already has pre-synced data, Sync should just index it and do not transfer files (directory structure should be completely same).I am trying to do the same thing, sync the Documents folder on my desktop, to the Documents folder on my laptop. I have tried doing it in both directions, but what happens in both cases is that it creates a duplicate Documents folder under Documents. My desktop is Windows 7 and my laptop Win 8, is the problem with the directory structure? Quote Link to comment Share on other sites More sharing options...
RomanZ Posted December 20, 2014 Report Share Posted December 20, 2014 @Michael HortonCould you please let me know exact path you try to add? Many user folders in Windows are actually not folders but Directory Junctions. Sync does not work well with Junctions. So right solution here would be to add real path Junction points to. Quote Link to comment Share on other sites More sharing options...
pa.ck Posted March 10, 2015 Report Share Posted March 10, 2015 Indexing completed. Both systems show 1 of 1 Peer, but out of sync with the full load of files to be transferred, but not transferring anything. Actually, when I checked both peers show each other as offline. They are both certainly online (I'm looking at both of them at the same time on the same Lan, can ping between them etc). Well shoot, this is a whole new problem now... But still, it seems as though they have communicated enough to know they are out of sync, and both are showing needing to send or receive the entirety of my content. @Triumph23Did you ever figure out why it was trying to sync all the data even though the folders were identical?I'm having the same issue with v1.4.111. Apparently it's going to take 5 months to sync even though the directories are identical. Quote Link to comment Share on other sites More sharing options...
Triumph23 Posted March 10, 2015 Author Report Share Posted March 10, 2015 Unfortunately no. I ended up bailing on using Sync to get this done, and went with an FTP syncing script that runs @reboot in crontab. Less elegant but it gets the job done. Unfortunately my system is deployed and I cant get to it for another couple of months. Once I do, I am looking forward to trying Sync 2.0. This project is such a cool idea, hopefully the kinks will get ironed out soon. At the time of writing my first question I believe Sync 2.0 was in alpha and unavailable for a Pi (without a headache). Haven't researched thoroughly yet, but I am HOPING there is a simple repo install, or at least an easily downloadable package to install 2.0 on a Raspbian machine. Quote Link to comment Share on other sites More sharing options...
pa.ck Posted March 10, 2015 Report Share Posted March 10, 2015 Thanks for the feedback. Quote Link to comment Share on other sites More sharing options...
12345lamacun Posted June 4, 2015 Report Share Posted June 4, 2015 Is this working within Sync 2.0?Copying a beta version of the folders content to the second peer, add it as read-only, and it transferrs the new files and delete the ones i deleted on the owner-peer? Quote Link to comment Share on other sites More sharing options...
RomanZ Posted June 5, 2015 Report Share Posted June 5, 2015 @12345lamacunYes, it does. Although, if some files are newer on your owner-peer they won't be replaced unless you ensure the checkbox "Overwrite changed files" is checked in folder properties of RO peer. Quote Link to comment Share on other sites More sharing options...
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.