benplumley

Takes Forever To Load List Of Sync Folders

14 posts in this topic

When I turn on my computer, it can take up to 20 minutes for Sync to get past the 'loading list of sync folders' stage. I only have 3 synced folders with a combined total of under 20GB. If I kill and restart Sync, it takes just as long.

Share this post


Link to post
Share on other sites

benplumley,

 

How many files do you have in these folders? Are these files big? What is your PC OS? 

 

When BitTorrent startsб it reads info about synced folder/files from sync.dat file. Is sync.dat in your case available for BitTorrent Sync? It is located in: 

Windows: %APPDATA%\BitTorrent Sync\sync.log
Mac: ~/Library/Application Support/BitTorrent Sync/sync.log
Linux: ./sync/sync.log

 

 Did you try re-adding the folders to BitTorrent Sync? 

Share this post


Link to post
Share on other sites

Total size 18.24GB, total number of files 15583. I'll try re-adding the folders. I have sync.dat and I'm able to open it, so I'd assume BTSync can open it too.

 

I have another problem as of yesterday that's causing all files to be unable to sync. It affects all my peers. Here are the relevant debug log lines: 

redacted, will reupload

The last line is repeated once per second. The file never gets synced, and nor do any other files I add. The debug log doesn't mention any other files, just Past Papers.

Edited by benplumley

Share this post


Link to post
Share on other sites

Here are the logs.

[2014-06-14 20:26:37] Fully synced with 10905A0FF843F7039A4DFCA735465265614E490A[2014-06-14 20:26:37] Merge: processing nodes message for /[2014-06-14 20:26:37] Merge: will request nodes for /Barton[2014-06-14 20:26:37] Merge: sending get_nodes message[2014-06-14 20:26:38] Sending broadcast ping for 4 shares[2014-06-14 20:26:38] Merge: processing nodes message for /Barton[2014-06-14 20:26:38] Merge: will request files for /Barton[2014-06-14 20:26:38] Merge: will request nodes for /Barton/Past Papers.xlsx[2014-06-14 20:26:38] Merge: sending get_nodes message[2014-06-14 20:26:38] Merge: processing nodes message for /Barton/Past Papers.xlsx[2014-06-14 20:26:38] Merge: will request files for /Barton/Past Papers.xlsx[2014-06-14 20:26:38] Merge: sending get_files message[2014-06-14 20:26:38] Merge: processing files message with 2 files[2014-06-14 20:26:38] Merge: Local file Barton is older (1402769804) than remote (1402773817)[2014-06-14 20:26:38] Merge: Local file Barton\Past Papers.xlsx is older (1402666636) than remote (1402773817)[2014-06-14 20:26:38] SyncFilesController: Got 2 files from remote (192.168.0.20:33088)[2014-06-14 20:26:38] SyncFilesController: Got file from remote (192.168.0.20:33088): Barton state: 1 type: folder total:0 have:0 t:1402773817 mt:1402773998 0000000000000000000000000000000000000000[2014-06-14 20:26:38] SyncFilesController: Got file from remote (192.168.0.20:33088): Barton\Past Papers.xlsx state: 1 type: file total:2 have:2 t:1402773817 mt:1402773794 6004C0E6A150A68A8457A2CDD63F3FF15C21A89C[2014-06-14 20:26:38] SyncFilesController: remote file Barton\Past Papers.xlsx is newer and different from local. Removing local torrent.[2014-06-14 20:26:38] State sync finished for folder \\?\C:\Users\Ben\Documents[2014-06-14 20:26:38] Tree hash changed while merging, cancelling sync with 10905A0FF843F7039A4DFCA735465265614E490A[2014-06-14 20:26:39] LoadTorrent: requesting to load file Barton\Past Papers.xlsx which is modified on disk (cur:1402773794 stored:1402773794 size1:59789 size2:59789 hasInfo:1) - rejecting until file info is updated[2014-06-14 20:26:40] LoadTorrent: requesting to load file Barton\Past Papers.xlsx which is modified on disk (cur:1402773794 stored:1402773794 size1:59789 size2:59789 hasInfo:1) - rejecting until file info is updated[2014-06-14 20:26:41] LoadTorrent: requesting to load file Barton\Past Papers.xlsx which is modified on disk (cur:1402773794 stored:1402773794 size1:59789 size2:59789 hasInfo:1) - rejecting until file info is updated 

I just re-added the folder, and it's completely broken it. There are ~15000 files in this folder and it indexes 32 files then stops. I'll try reinstalling it.


Update: it's indexing them now.

Share this post


Link to post
Share on other sites

@benplumley

 

Okay, what about initial issue? Are files re-indexed 20 minutes or it is done in a prompt manner now?

Share this post


Link to post
Share on other sites
I have the same problem. I'm running Windows 2008 R2 on both ends on a LAN.
 
2 shared folders
 
1 folder with 6.1GB in     34,330 files
1 folder with 1.9TB in 1,606,693 files
 
Both client versions: 1.9.109
LAN Encription is false
Disk low priority is false
Force LAN TCP is missing (was this setting removed?)
Debug Logs is empty with just the standard FFF

Share this post


Link to post
Share on other sites

@bradbamford

 

Under "same problem" I guess you have the topic name issue - folders loading takes a lot of time. 

1) What is the amount of memory on your server? 1.6M of files is large amount and will consume lots of memory.

2) Is sync.log completely empty? There should be some data even when debug logging is off. Could you please verify it? note that debug.txt should contain "FFFF" instead of 3 Fs (just in case it was not a typo).

Share this post


Link to post
Share on other sites

@bradbamford

 

Under "same problem" I guess you have the topic name issue - folders loading takes a lot of time. 

1) What is the amount of memory on your server? 1.6M of files is large amount and will consume lots of memory.

2) Is sync.log completely empty? There should be some data even when debug logging is off. Could you please verify it? note that debug.txt should contain "FFFF" instead of 3 Fs (just in case it was not a typo).

 

Happens on both ends.

Each box has 24GB ram.

On average, BTSync.exe consumes 900,000 K memory.

Total memory use is always under 35%, CPU under 25%.

Yes, sync.log works just fine, but the debug.txt never shows any errors.

To test, I blocked BTSync in Windows Firewall. That added entries to debug.txt, but the "folders loading takes a lot of time" issue never shows any errors. It's just slow.

Share this post


Link to post
Share on other sites

I have the same problem, loading Resilio Sync takes forever, or at least too much to handle the syncing.

474 080 files in 97 336 directories, total of 36 Gb.
Windows 7 Pro on both sides.

Could you put efforts to optimize loading of the files?
Or do it in background, because the shared directories are not even displayed, before Loading is not finished.

Thanks!

 

On 2014. 06. 13. at 2:57 PM, Helen said:

benplumley,

 

How many files do you have in these folders? Are these files big? What is your PC OS? 

 

When BitTorrent startsб it reads info about synced folder/files from sync.dat file. Is sync.dat in your case available for BitTorrent Sync? It is located in: 

Windows: %APPDATA%\BitTorrent Sync\sync.log
Mac: ~/Library/Application Support/BitTorrent Sync/sync.log
Linux: ./sync/sync.log

 

 Did you try re-adding the folders to BitTorrent Sync? 

 

 

 

Share this post


Link to post
Share on other sites

are you sure this is the same problem? The tread is almost 3 years old, and since then Sync has been updated quite a few times. What Sync version you have now? 
If it takes too long to load, then this might be just loading database to RAM (it's been discussed here on forum multiple times). Check size of .db files in storage folder 

Share this post


Link to post
Share on other sites
4 hours ago, Helen said:

are you sure this is the same problem? The tread is almost 3 years old, and since then Sync has been updated quite a few times. What Sync version you have now? 
If it takes too long to load, then this might be just loading database to RAM (it's been discussed here on forum multiple times). Check size of .db files in storage folder 

Using Resilio Sync v2.5.5 on both side.
The used memory by Resilio Sync is about 1.3Gb, using 25% of 4 processors, so 1 processor is running at 100% (which is used by Resilio Sync).
Total physical memory is 8Gb, 85% is used now.

Also I get very often app has "Not responding" State in task list, for long time.


Likely it is the same problem, as I use Bittorrent Sync since around v1.2 - 1.3 version. About since 2014, 3 years :-)

I always had this problem, but in earlier versions it was not so bad.
Perhaps as the file number increases, the loading time & indexing time increased.
But now it makes Resilio Sync unusable. I have to wait several hours to become responsive & finish loading, then do something.
The 474 080 files in 97 336 directories, total of 36 Gb. should be not much for an online backup software, like Resilio Sync.

Today, after several hours I reached the state, that I was able to copy key, then paste into other side.


EDIT2:
Total size of .db files stored in c:\Users\user\AppData\Roaming\Resilio Sync\: 5,65 Gb.

EDIT3:
I will try to add to the .sync\IgnoreList of one backup folder, which contains Mozilla profile, to exclude the cache:
\Mozilla_Profile\Profiles\myprofile\thumbnails\
\Mozilla_Profile\Profiles\myprofile\cache2\

 

Share this post


Link to post
Share on other sites

oh, RAM usage, database have been discussed here on forum a very lot times. Sync on start reads the database  and loads them to RAM. the bigger database you have, the longer it takes. 

Syncing other apps cache and their service files is indeed not a good idea - these change quite a lot, new ones are created -> database grows bigger, more RAM taken, more resources are necessary to manage this folder - the folder tree is really big.

Apart from ignoring the files, you can remove the share from Sync and add it back so that database gets cleared. Also, increate rescan interval in power user settings. 

Share this post


Link to post
Share on other sites

Thanks for your reply & ideas.

I would not like to re-add the folders, because syncing 35-40Gb would take a long time.
I will only do this, if this would clear such big database items, which would not be cleared after a time.

Yesterday I increased rescan interval from 2 hour to 6 hour (21600).
Also, yesterday I upgraded from Resilio Sync v2.5.5 to Resilio Sync v2.5.6.

Currently the sync is going, but two folders (10 Gb and 5 Gb, both having  a lot files, are progressing very slowly, currently they are at 4%).

 

Share this post


Link to post
Share on other sites
22 hours ago, Webmaster33 said:

because syncing 35-40Gb would take a long time.

there's no need to resync all the files, you can use the folders with already synced files.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now