Sync Not Indexing Some Files?

[ficedula]

I've installed btsync on my laptop (Windows 8) & a Linux server. I have two folders shared on the laptop - the server has them synced as RW although in practice I'm normally only changing files on the laptop.

One folder (~5000 files) is synced fine. The other is listed in sync on the laptop as 1.4GB in ~13000 files, but it's actually about 4GB in 25,000 files (both Explorer & Command Prompt confirm), and the Linux server has only got 1.4GB of the files.

 

Interestingly the Linux server has files/directories with filenames beginning with letters L-Z and only one file beginning A-K (there should be hundreds!) so it's like the indexing on the laptop has just not finished ... but the laptop claims it's indexed OK (and I set it up last week so it's had a good few days to do so!).

 

The only entries in the .SyncIgnore file on the laptop are the following which look like reasonable defaults (I haven't intentionally changed the ignore file):

.DS_Store.DS_Store?._*.Spotlight-V100.Trashesehthumbs.dbdesktop.iniThumbs.db

Laptop is running sync 1.3.94. Does anybody have any suggestions for how to fix, or how to get more info to help diagnose things? Thanks!

 

(EDIT: I just tried creating a new file in the folder named AAAA.txt and that's synced fine, so it looks like new files are syncing fine, it's just failed to index [some of] the existing files when I created the share, and isn't adding them in subsequently?)

 

(FURTHER EDIT: I enabled logging, and the following lines look relevant; it gets halfway through scanning the folder and then gives this error;

[2014-05-12 11:42:35] SyncFilesController [file updated]: Processing file \\?\C:\Users\ficedula\Documents\My Videos 1399830078 4096[2014-05-12 11:42:35] SyncFolderScanner: GetFileList failed for folder \\?\C:\Users\ficedula\Documents\My Videos - error -1

...so I guess there's something in the My Videos folder that makes it fall over.

 

I'll try excluding that folder from the sync - because I don't need it synced anyway - but it'd be good if an error scanning that subfolder didn't prevent all the other subfolders/files from getting scanned/synced?

Edited May 12, 2014 by ficedula

[RomanZ]

@ficedula

Which version of BTSync do you use? This was a common issue in 1.2 versions: the indexing got stuck when it stumbled upon folder\file it can't read by some reason. 

 

If you are using 1.3, not 1.2 - can you please share the logs so we can take a peek what is going wrong? Send them to syncapp@bittorrent .com, also refer this topic in message body.

[ficedula]

Yes, I'm using 1.3.94 - email sent.

 

FWIW, the directory in question was actually a junction point Windows had created (redirecting the 'old' location to the new Music folder location for backwards compat, I guess?). Adding the directory in question to .SyncIgnore didn't seem to help at all, however removing the junction has now let the sync succeed and it's got the full file list / sizes indexed now - so the error isn't actually occurring for me any more (although I suppose I could add the junction back in if you needed to to try and recreate the error!).

 

Thanks

[RomanZ]

@ficedula

 

Thanks for the logs. Seems that would be enough to fix the issue.

[knwpsk]

I might have found something useful, or else a related edge case.  

Was having some problems syncing a large directory that was previously sync'd.  On both machines, I deleted the folder from Btsync, then re-added the same directory on both.  

 

I observed that on both machines, it *started* to index the directory, and it indicated it had finished, but apparently didn't finish.  On both machines, Btsync showed the folder to have N files with a total of N size; it had different numbers for that directory on each machine, and both were very very wrong numbers (much smaller than the actual directory).

 

My suspicion is that once btsync started to exchange data between the two machines, it stopped indexing prematurely.

 

So, now, I'm doing the same process again.  But this time, I only re-added the directory on ONE machine, and am waiting for it to fully index that machine before I re-add the directory to the other machine.  So far it seems to be successfully re-indexing the whole directory without trouble -- it has indexed 8x more data than it reported last time.

 

If my suspicion is correct -- that starting the transfer of data prematurely stopped the indexing process -- then this should probably be considered a "bug" that needs to be coded around.  The app shouldn't start transferring data until the indexing process has completed successfully.  Otherwise, one would see the problem that I saw:  it would start trying to upload/download files that already exist on the peer machine, and encountering errors or version problems.

 

(Update:  my second attempt fixed the problem, when I allowed one machine to fully index before re-adding the directory to other machines.  All machines are now fully synchronized, showing proper counts and file sizes, and not trying to overwrite each other with existing files.)

[dershope]

I'm having a similar issue.  When I add the "documents" folder to the Sync list it indexes only about 6mb of the 5GB of files.  This is using version 1.3.106.  When other folders are selected it indexes them fully.

[RomanZ]

@dershope

 

There might be folders or files where sync can't access due to permissions or if files / folders are open exclusively by another app. I suggest collecting debug logs and sending to me for analysis.