Frequent and long indexing stressing HDD & CPU


Recommended Posts

I use BTSYNC Pro on a windows PC, but the shared 70GB directory is on a NAS. Because of that I can't use window's OS notification system for indexing file changes. The work around is to set scanning intervals using  rescan interval in Sync settings -> Advanced -> Power user. I've set it to 36000 which is fine for me. This has been working fine on version 2.2.5

Recently I've updated to the latest 2.3.3 and my NAS was giving heat warnings. Finally, found out BYSYNC was indexing very frequently and for prolong periods of time causing my HDD to spinup. 

Temporarily I've downgraded back to 2.2.5 and indexing is back to normal, but I still want to keep updated.

Is there a workaround fix or am I doing something wrong? 

 

cpu_chart.png

Link to comment
Share on other sites

Quote

Recently I've updated to the latest 2.3.3 and my NAS was giving heat warnings. Finally, found out BYSYNC was indexing very frequently and for prolong periods of time causing my HDD to spinup. 

I'm having trouble with BtSync indexing frequently consuming more  than 70% of CPU after upgrading to 2.3.3 (296). I'm running on Mac OS X 10.11.3.

Can I provide more information to help you track this issue? Is this already on your radar? Or it's a better idea to downgrade until a proper fix?

Link to comment
Share on other sites

That is funny. If that is their proposed solution, I have a better one: quit the application. Why keep it running if it will not index and sync files?

For now I'm trying to use older versions, looking for a more stable one. But I think it's time for another round of searches for BtSync alternatives.

Link to comment
Share on other sites

I'm using Freenas and it doesn't support file change notification. I need to use rescan interval; I am very aware of this. I've been doing it with Version 2.2.5 for a while now fine. Indexing works and minimal load on the freenas.

The problem comes when upgrading to 2.3.3. Indexing takes a very long time about half a day as indicated by the graph. Downgrading back to 2.2.5 everything is normal again. 

Link to comment
Share on other sites

I am looking at the 500MB+ of sync.log.xxx.zip (that is a lot of compressed logs )

Appears BTSYNC 2.3.3 is indexing my files over and over again for hours and hours, almost all day. Something is wrong.

Uninstalled 2.3.3, wiped userdata and installed 2.2.7. Added my 70GB folder to index and it finished in 8 minutes. 

 

Link to comment
Share on other sites

Can anyone point to a safe link to download 2.2.7? BitTorrent Sync official site has AFAIK only 2.3.3 and - frankly - should remove it soon or at least leave older versions available. I thought this problem was due to recent upgrade to El Capitan but I see is a bug everywhere.

Link to comment
Share on other sites

2 hours ago, peogbert said:

Can anyone point to a safe link to download 2.2.7? BitTorrent Sync official site has AFAIK only 2.3.3 and - frankly - should remove it soon or at least leave older versions available. I thought this problem was due to recent upgrade to El Capitan but I see is a bug everywhere.

forum.getsync.com/topic/41708-latest-desktop-build-227

Link to comment
Share on other sites

I have had nothing but problems with the 2.3.x release branch. Every version has had the "indexing forever" bug, with neverending high CPU usage. If I forget to quit BitTorrent Sync when I'm on battery it's drained in no time. The changes made from 2.2 to 2.3 introduced some critical bugs to core functionality, and it makes me second guess my purchase of a pro license.

Link to comment
Share on other sites

Same Problem here. Since I upgraded, btsync is scanning on my Disc around the clock. And "Setting rescan interval to 0" is not a satisfying answer to me, because my NAS is also an active peer on downloading ISOs and other stuff. Feeling a bit like a beta tester...

mapper_truecrypt2-week.png

Link to comment
Share on other sites

On 12 March 2016 at 6:03 PM, KawateTadako said:

forum.getsync.com/topic/41708-latest-desktop-build-227

Thank you KawateTadako!

And:

56 minutes ago, generator said:

Some problem, has downgrading to 2.2.7 worked for other people and is it safe to downgrade?

After downgrading to 2.2.7 it indexed in few minutes all what it has tried to index for hours (if not days) and it is syncing as a charm.... (OSX El Capitan 10.11.3 here)

Link to comment
Share on other sites

@KawateTadako @MaBu81 We've recently found one more possible root cause.

Starting from 2.3 we've fixed notifications over SMB (if SMB server and client support SMB 3.0, which seems to be your case, as logs contains number of notifications pending). All notified changes are pending the rescan and prolong the indexing. Could you please try disabling the "enable_file_system_notifications" power option on your NAS and see if your indexing will take the same amount of time as in 2.2.7?

@repertor @gabled-bravado-dell @generator We need logs to see what happens on your machines. In general, indexing in 2.3 got kind of lower priority than other tasks performed by Sync, therefore it may last slightly longer, but it should not take several times longer.

Link to comment
Share on other sites

@RomanZ my FreeNAS-9.3-STABLE is running SMB2, SMB 3.0 isn't supported. I upgraded to 2.3.3 again, set  "enable_file_system_notifications  to false. Indexing still taking ongoing after an half hour. On 2.2.7 it only takes minutes to complete.

Looking at sync.log I see OnNotifyFileChange, Journal: assign file_id, will scan folder, entry, erase file_id lines repeated for the same files over and over again. Sync.log overflows, get archived as *.zip until I have GB's of log stored.

Link to comment
Share on other sites

On 3/16/2016 at 0:27 AM, RomanZ said:

@KawateTadako Investigation of your log has shown that file notifications could be the reason of indexing slowdown. I wonder if you could send us fresh logs after you disabled notificaitons? Please also mention this forum topic so support engineer who takes care of you will be aware of it. Thank you!

Updated to 2.3.5, still having the same issue. Left it indexing for 40 minutes before giving up and reverted back to 2.2.7 which index completed in minutes. 

uploaded new logs to ticket request #40669

Link to comment
Share on other sites

On 15.3.2016 at 2:26 PM, RomanZ said:

@KawateTadako @MaBu81 We've recently found one more possible root cause.

Starting from 2.3 we've fixed notifications over SMB (if SMB server and client support SMB 3.0, which seems to be your case, as logs contains number of notifications pending). All notified changes are pending the rescan and prolong the indexing. Could you please try disabling the "enable_file_system_notifications" power option on your NAS and see if your indexing will take the same amount of time as in 2.2.7?

*SNIP*

Seems so be the reason for me. Removing the only Samba directory I use in common with btsync stopped the endless indexing. Trying out to set the "enable_file_system_notifications" when Im home again.

Thank you so far.

Link to comment
Share on other sites

I seem to have a similar problem on some of my computers after updating from 2.2.7 to 2.3.5. The indexing on 2.3.5 (365) seems quiet frequent (on most of my devices) and taking long compared to 2.2.7. It's not stopping to index at all on my "Server".

___

- On my Workstation (Win 8.1, i7-2600k, 24 GB RAM) with a subset of all my folders (19 folders, ~360 GB), everything seems to work like before. CPU (0-0,2% usage when not indexing and closed UI) and RAM usage (~190 MB) of BTsync is normal, indexing is finished quite fast stressing one logical CPU core (12,5% CPU usage). 

- On my Notebook (Win 8.1, Thinkpad Yoga S1, i5-4200u, 8 GB RAM) with most of my folders (22 folders, ~600 GB), it seems BTsync is constantly doing something. CPU (almost constantly around 0,2-2% usage when not indexing and closed UI), RAM usage (~250 MB) indexing takes quite long and stressing again one logical CPU core (25% CPU usage). This behavior is quite annoying, since it's draining my battery quite fast!

- On my "Server" (Win7, Core2Quad Q6600, 3 GB RAM) with all of the folders (26 folders, ~890 GB) BTsync is constantly indexing the same four folders that are syncing with my sisters MacBook Pro (I have to check which version of BTsync is running on that device) even though her Notebook is offline at the moment. CPU usage (constantly 12-15% usage) and RAM usage (~830 MB) is also quite high, but no wonder if it's constantly indexing. Is there anything I can do to stop the constant indexing of those four folders?

- On my Android Phone (Samsung Galaxy S4 i9505, 5.0.1) everything is finally working fine since the last update to 2.3.5 (480), since before that it was constantly indexing there. Even syncing to the 128 GB SD Card! 

- On my sisters MacBook Pro (still have to check the BTsync version there), with a small subset of the folders (4 folders, ~260 GB), I have no idea if BTsync is running properly. 

___

I guess I should have stayed with the old version (2.2.7) as well. But before I'm going to revert to that one, I'd like to know if there is some other solution that I could try. Should I send logs? If yes, from which devices? How big should my log file size be with 890 GB of files?

Link to comment
Share on other sites

I've updated my workstation, server and notebook to 2.3.6, but I still seem to have the same problem. It's quite annoying that the server is constantly indexing the folders. I assume it's due to the folder count and size compared to the other machines, which just have a subset of those. The CPU and HDD of the server are never idle due to the indexing. Even more annoying is the reduced battery life on my notebook due to the frequent indexing of Sync. 

Feel like rolling back to 2.2.7, but before that I'll send you the log files of my server, maybe that will help solving the issues. 

I guess I'll just paste the link to this thread in the help request?

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.