ms2oo8

Indexing After Re-Starting Sync Client

Recommended Posts

What is the bahviour of Bittorrent Sync client?

 

I just finished an initial sync of 2.1TB of files sitting on a NAS, with the Bittorrent Sync client installed on a Windows 7 desktop running Sync 1.4.93.

 

The inital indexing took about 12 hours, and now that I've restarted the client the status says Indexing again. Can I expect a 12 hour Indexing period any time I restart a client with this amount of files?

 

My impression was that Indexing would happen one time, and be smart enough to know the files have been Indexed even after Bittorrent Sync restarts?


Also if this really is the case, is there a way to tell how much progress the Indexing has made? Initially, the file of the files grew, and I could tell how far along it was. Now that the files are indexed, the size alraedy shows 2TB and it's indexing.

Share this post


Link to post
Share on other sites

My impression was that Indexing would happen one time, and be smart enough to know the files have been Indexed even after Bittorrent Sync restarts?

Sync needs to update its index each time it starts - otherwise it has no way of knowing if the contents of the folder's its monitoring have changed in any way whilst it's not been running!

 

This "index update" process shouldn't take anywhere near as long as the "initial index" process when you first added the folder to Sync

 

is there a way to tell how much progress the Indexing has made?

Not at present, Sync will just indicate "Indexing...". I suppose ideally the Status or Progress column should indicate how far through the indexing process Sync is... so perhaps this is a suggestion for the Feature Requests forum.

Share this post


Link to post
Share on other sites

ok thanks for the quick response.

 

It's still Indexing but it seems like it's reset again, and is around 500 GB and counting up again

Edited by ms2oo8

Share this post


Link to post
Share on other sites

Sure if I can figure out how to send it :)

Because the initial sync finally finished, now I've added 5 MB of files to the one NAS, and it's sitting quite awhile in Indexing mode again, just for that 5MB of new files added.

Share this post


Link to post
Share on other sites

@ms2oo8

Our support still did not manage to get logs from you. Once you provide us logs - we'll start analysis ASAP and hopefully will find out what makes your Sync indexing all the time.

Share this post


Link to post
Share on other sites

How many files are you trying to sync?

 

I heard on a podcast,that someone tried to sync there giant family photo library that contained tens of thousands of files. That it just killed their home server because it had to rescan every single files over and over again. But this was when BTsync first came out. So I bet there has been improvement.  

Share this post


Link to post
Share on other sites

How many files are you trying to sync?

 

I heard on a podcast,that someone tried to sync there giant family photo library that contained tens of thousands of files. That it just killed their home server because it had to rescan every single files over and over again. But this was when BTsync first came out. So I bet there has been improvement.  

 

 

:D  :D  :D

Even Boeing 747 can't kill a good server :)

 

In all modern operating systems all frequent read\write operations on HDD are cached in computer RAM. And each HDD has internal cache too! So don't worry about "rescan operations". Windows Explorer doing this always for a lot of system files right after powering on the pc. ​

http://en.wikipedia.org/wiki/Disk_buffer

http://msdn.microsoft.com/en-us/library/bb742613.aspx

http://www.wikihow.com/Change-Process-Priorities-in-Windows-Task-Manager

Share this post


Link to post
Share on other sites

it's 2TB of data. (628,000 files in 30,311 folders)

I have the Sync app installed on a Windows boxes on either end, (i5 750 x64 Win 7 with 8GB RAM and a i7 2600 x64 Win 7 with 8GB RAM)  and both are not performing any other tasks.

 

I reconfigured the Peers to map via a drive letter vs the UNC path. Everything synced up, except 1 open office file, which was locked due to it being open. When I closed it, sync stayed stuck at the locked file status. So I closed the app on both ends again, and now we're back to Indexing . . . .


having said all that, it shows Indexing status as we speak, but it seems to be syncing any changes I make immediately right now

Share this post


Link to post
Share on other sites

So I think I am seeing something similar which I am trying to get help w/ in ticket #32426-btsync-appears-to-be-indexing-ignored-files.  My problem is I have a large number of files generated each hour ( logging ) which seems to have to be indexed and takes hours.  the ignoreSync pattern seems to be working by not syncing these files but unfortunately is seems to be indexing them regardless of that config... Its quite disappointing :( 

Share this post


Link to post
Share on other sites

@ms2oo8

What happened? From your previous post it looked like Sync was doing its job.

I think it is, but it seems to Index almost permanently. I'm concerned that it takes so long to Index changes because I'm afraid that it won't keep up.

Share this post


Link to post
Share on other sites

@jdyer

It looks to be different case. In your case Sync is simply indexing files in folder that is added to ignore list (which is quite normal as folder is already in the tree), while in @ms2oo8 case indexing never completes.

 

@ms2oo8

If you are willing to help us in debugging this issue - well need a debug logs from your computers.

Share this post


Link to post
Share on other sites

So I did some testing with RomanZ and it seems like my install is stuck completely re-Indexing almost constantly. We haven't figured out why yet, but think it might be due to the folder locations being mapped drives on SMB network shares.

 

It keeps re-indexing previously indexed files, that I am 100% certain haven't been changed by a person.

 

This leaves a few scenarios possible

1) software on the NAS box is reading files (for whatever reason, rsync backup of the NAS box to another NAS box etc) and is triggering Sync to re-index somehow

2) a bug in the Sync software as expained above

 

I'd be inclined to think it's a bug, since I have another NAS with a mapped SMB share on the other end, with the same issues, but it's a different brand of NAS box, with no backup configured on it.

Share this post


Link to post
Share on other sites

@ms2oo8

I don't have exact answer for you as we haven't found the root cause in 1.4 yet. There are numerous core changes in 2.0, so it might happen that issue was already resolved - and we'll find that our only when we find your issue root cause.

Share this post


Link to post
Share on other sites

Is this bug being tracked in your system? Also when is 2.0 ready for testing?

 

As RomanZ has indicated, they are aware of the issue: "we are working on it, will update when get some news" and "I don't have exact answer for you as we haven't found the root cause in 1.4 yet"

 

Also when is 2.0 ready for testing?

It'll be ready when it's ready - if you've followed the "Sign Up" button in this blog post, you'll be on the list and be amongst the first to know once 2.0 becomes available for testing.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.