Sync Your Windows Profile the Smart Way!

[yottabit]

Running Windows 7 here. My profile directory contains 52,125 files totalling 52.1 GB.

A large portion of those files are Java caches, web browser caches, temp files ... all things that would be stupid to sync.

The only subdirectories I'm interested in syncing (Documents, Desktop, Downloads, etc.) contain only 10,206 files totalling 29.6 GB--only 20% of the files and 57% of the data. There is an easy way to put all of these subdirectories into a single directory so that a single folder share can be made with btsync (instead of an individual share for each one of the interested subdirectories).

NTFS 3.0 and higher (Windows 2000) support Directory Junctions, which are very similar to symlinks (symbolic links) in UNIX. In Windows 6.x (Windows Vista and Windows "7") you can create Junctions from the command line with the "mklink" utility, included already with Windows.

Using this utility, I can create Directory Junctions, of only the subdirectories I'm interested in syncing, into a new subdirectory. I then create a single btsync share of that new subdirectory which now encompasses all of the other directories--thanks to Directory Junctions--without actually being a "copy" of the data that would need updated.

I wrote a little command script that does this for you. Copy this text into a new text file, then rename it "btsync-win7-profile.cmd", after which you can double-click it to run.

I also attached the file directly to this post to make it even easier.

@echo off

REM mklink script for Win7

REM by Jacob McDonald, btsync@McDonaldTechnology.com

REM version 130424a-jmmcd: initial version

REM version 130425a-jmmcd: removed extraneous 'echo' in description

echo.

echo This script creates a new subdirectory called "BTSync" "in your

echo user profile subdirectory. This will be located here:

echo.

echo %userprofile%\BTSync

echo.

echo By default this script links in Contacts, Desktop, Documents,

echo Downloads, Dropbox, Favorites, Music, Pictures, Saved Games,

echo and Videos.

echo.

echo ******************************************************

echo *** ***

echo *** This probably only makes sense with Windows 7! ***

echo *** ***

echo ******************************************************

echo.

echo If you want to use it on a different Windows version, or if you

echo want any other subdirectories not listed above, please edit this

echo script first. It's pretty self-explanatory.

echo.

echo Ready to begin. Press Ctrl+C (then answer Yes) if you want to stop.

pause

cls

cd %userprofile%

mkdir BTSync

mklink /J "BTSync\Contacts" "Contacts"

mklink /J "BTSync\Desktop" "Desktop"

mklink /J "BTSync\Documents" "Documents"

mklink /J "BTSync\Downloads" "Downloads"

mklink /J "BTSync\Dropbox" "Dropbox"

mklink /J "BTSync\Favorites" "Favorites"

mklink /J "BTSync\Music" "Music"

mklink /J "BTSync\Pictures" "Pictures"

mklink /J "BTSync\Saved Games" "Saved Games"

mklink /J "BTSync\Videos" "Videos"

echo.

echo Finished! Now create a new BTSync share of the following location:

echo.

echo %userprofile%\BTSync

echo.

echo Please note: you can delete the entire directory above at any time

echo without affecting any of the data to which it links.

echo.

REM btsync will have CLI option support in a future version; then you can

REM hopefully launch it to share the subdirectory above automatically at

REM completion!

pause

Enjoy!

btsync-win7-profile.zip

[yottabit]

I just wrote a reply to another thread where a user wanted to share individual files, not folders. Of course BTSync isn't designed to handle this scenario, but there's always a work-around, once again using features stolen copied borrowed from UNIX filesystems now implemented in NTFS. I wanted to post into this thread because it's so closely related to my script above.

In this case, rather than linking in whole subdirectories, we hardlink individual files into a directory that's shared. Three huge advantages to this method:

1. Doesn't use additional diskspace because these aren't copies but simply additional filenames that point to the exact same data on the disk, and
   
2. If you update/modify any one of the hardlinked files, all of the others reflect the same changes (again, because they all point to the exact same data on the disk), and
   
3. After hardlinking, you can feel free to delete any of the files you like (even the original one!) and all of the other hardlinked files will still contain the exact same data and relationship.
   

In other words: Neato! Here's the scoop:

Here's another way! If you're using Windows 6.x (Windows Vista or Windows "7"), use hardlinks to keep the hassle to a minimum.

1. Create a new folder that you will share with BTSync.
   
2. Drop to CLI and use "mklink /H" to hardlink the files you want to share into this new folder.
   
3. Drink a beer and relax!
   

So here's the trick. If you aren't familiar with UNIX, a hardlink is simply another filename pointing to the same data on the disk. So let's say you have a file called MyBigFile located on your Desktop. Well, you don't want to share your whole Desktop, but just that one file. So we do this:

1. Create new folder on Desktop called SharedFolder.
   
2. Open a CLI window (Start -> "cmd" at the bottom -> click cmd.exe in the search results at the top).
   
3. cd %userprofile%\Desktop\SharedFolder
   
4. mklink /H MyBigFile ..\MyBigFile
   

​That's it!

You can use this method to even change the name of the file, but not duplicate the data like a copy would. And with a hardlink, now both files are just pointers to the same data on the disk, so you can feel free to delete one without damaging/deleting the other. For instance, if you hardlink file1 to file2 and file3, you can delete 1 or 2 of those (even file1!) and the others will still work just peachy. And just to be clear, if you edit one of those files, the updates will appear in the others as well (because like I said, they're just different names but pointing to the same data on the disk).

Oh, and as usual, Microsoft implements features into Windows without providing any method of using them from the GUI. Some of these features even require third-party utilities! So sorry about the CLI if you're not comfortable with it, but this is the easiest way to go. There is also a third-party GUI utility called NTFSlink that was used prior to Windows 6.x to provide access to this function before Microsoft even had a CLI utility. I don't know if using it in Windows 6.x is supported, but if you install it you can just right-drag files into the new folder, and from the context menu select "Create Junction Here" (for directories) or "Create Hardlink Here" (for files).

http://sourceforge.n...ts/ntfslinkext/

EDIT: I guess NTFSlink is no longer under development. Here's an alternative that also works with Windows x64: http://schinagl.priv.at/nt/hardlinkshellext/hardlinkshellext.html

[GreatMarko]

There is an easy way to put all of these subdirectories into a single directory so that a single folder share can be made with btsync (instead of an individual share for each one of the interested subdirectories).

...or you could just use .SyncIgnore to selectively sync!

So, if you wanted to sync your windows profile, but exclude say the folder "AppData", this could be easily done using .SyncIgnore - there's no need to create directory junctions / symlinks!

[yottabit]

Yup, that's another way. But I find that over time my profile directory becomes more and more full of cruft.

Some applications add their own junk in there without putting it in the proper subdirectory, and I have been known--on occasion--to throw small utilities in their myself!

My way is an opt-in method, whereas using the .SyncIgnore is an opt-out method. Both work. Both have advantages and disadvantages.

[GreatMarko]

My way is an opt-in method, whereas using the .SyncIgnore is an opt-out method. Both work. Both have advantages and disadvantages.

Yep - fair point! .SyncIgnore is indeed essentially an "exclusion" filter, rather than an "inclusion".

I do think editing a .SyncIgnore file is probably a little more "user friendly", and easier to keep track of though, than creating hardlinks/folder junctions/symlinks, etc - especially for those novice users who are perhaps not as computer savvy as the rest of us!

I'd always encourage the use of .SyncIgnore for "selective" sync wherever possible - but I do think it would also be a good idea for BitTorrent Sync to have the option for a ".SyncInclude" file instead - which would allow you to specify which folder/files should be included in the sync

[yottabit]

Yes, definitely easier for most novices, but then again they have to know how to edit a (hidden) text file to begin with.

At least with my script they can just run it and be done. The actual linking/junctioning is really hidden from them and as long as they understand that the new BTSync folder's contents are not copies but simply additional names to the same data, they should be good. Even if they delete one or the other, their data is still intact and safe.

[Dickel]

I've been experimenting with this idea of using 'junctions' in syncing 3 machines (2 on Win7 and 1 on Win8), using the excellent "link shell extension" program rather than the CLI.

In spite of doing a lot of searching & reading, I haven't been able to decide which type of 'junction' to use - "junctions" or "symbolic links". At the moment I have one folder using a "junction" and two other folders using "symbolic links". So far, I haven't seen any difference in the way they operate.

Any insight would be appreciated - thanks.

[yottabit]

Junctions seem to be more compatible and better supported.

[Dickel]

OK, thanks. I'll change the symlinks to junctions and see how it goes.

[GFIScripter]

I used your script (above) and modified it for my environment and this worked really well on the original version I downloaded (1.0.xx ). I had it working under Win 7 & Win XP but after I decided to upgrade to the latest version (1.1.15) My Windows 7 Computers do not want to work. Actually they do but with a fraction of the data that should be moved... have you experienced this?

Windows XP it works fine...

for full detail writeup: http://forum.bittorrent.com/topic/20794-windows-7-btsync-junctions/

Thanks

[yoshm]

Given that the idea of creating links to files or directories is based on  an idea from Unix/Linux - Is there a way to do this under Android (preferably that does not require root)? Since this is a bit of a diversion from the OP, I've created another topic to find the answer to this: http://forum.bittorrent.com/topic/25772-is-there-a-smarter-way-to-syncbackup-android-folders/

 

If you answer here, I'll copy it over to there, too.

 

Thanks,

Yosh