derBoris

Synology and 2 Macs - Optimize 'indexing'?

Recommended Posts

Hi,

i've installed BTSync on my Synology and on two Macs to sync around 100k files in about 25 GB (lots of JPG and RAW files and a Lightroom database). So far it works… but…

The Mac-Client – on is a MacBook Pro – is doing an 'indexing' every 10 Minutes. Every 'indexing' takes a few minutes where BTSync is consuming up to 100% CPU.

I've searched a bit and read some longer threads about this indexing-feature but i think i didn't really get it: 

- Indexing is building a catalogue of all files that i like to sync to detect new/deleted files?

- But then, changes made on one mac are transferred to the NAS in nearly realtime… not every 10 Minutes… And if i wake up my MacBook, the files are downloaded from the NAS nearly instant … not after 10 Minutes

So again - what is the purpose of creating an index every 10 Minutes? Are there save ways to optimize the clients behavior?

 

I like to have my Lightroom-Files in sync on both Macs and i'm using a NAS because both Macs are not running at the same time. Both Macs are creating/deleting files (JPG-Previews etc.). The NAS is running 24/7 so the 'truth' should always be on that NAS…

In this case: Is it save to say, that i don't need to build an index on the macs every 10 Minutes but for example only once a day? I mean: If i'm working on the local iMac, changes are synced to the NAS in real-time. If i'm working on the MacBook it's  the same when i'm at home.

Only if i'm working on the MacBook while i'm traveling i need to be sure, that the MacBook will sync all new/changed files to the NAS when i'm back home - While BTSync is running, it should know those changes in realtime, right? 

 

So - Why do i need such index in my use case? Is it save to turn it off or set it a way longer interval (once every 10 hours or so) on the mac-clients? Could i even change the interval on the NAS (or turn it off completely) to let the HDDs sleep again from time to time?

 

Thanks a lot for your help to clarify this topic :)

Share this post


Link to post
Share on other sites

@derBoris Sync detects changes via rescan (slow and reliable) and OS notifications (extremely fast, but unreliable). There are a number of edge cases when OS notifications may fail, therefore Sync uses both methods simultaneously. If you does not need Sync to happen more often than once a day - you can set rescan to once a day.

As for your NAS - if you don't ever change data on NAS (changing over SMB/CIFS also counts!) but only use it as device which always run and ready to seed data - you can disable rescan there at all. And in general, take a look at this article regarding letting NAS sleep :).

Share this post


Link to post
Share on other sites

Hey @RomanZ, thanks for the quick response.

1.) I never touch the Sync-Storage on my NAS directly. Only the two Macs are writing/reading from/to that directory (and this happens only via BTSync; Never via finder/terminal/...). So i think it's save to disable the indexing on the NAS.

2.) I need BTSync to sync changes to the NAS as quickly as possible (not once a day). But as far as i could see, BTSync is doing just that! If i'm working with Lightroom, changes are written almost in realtime to the NAS. So i would think that i never every need an indexing on the Macs too... I would just set it to a long interval for such edge-case... but then... not sure if i really need that at all - in my usecase...

a) the Lightroom-Database is synced right when it changes. Done. No Index needed. (and index wouldn't help too: If this file isn't synced to the NAS NOW, it wouldn't for the rest of the day because i set the Mac to sleep before it could index...)

b) the other files are JPG-Previews... if some of them wouldn't sync - no problem: The other Mac will rebuild them as needed (Lightroom)

 

My worst-case scenario:

- change the Lightroom-Database on Mac1

- BTSync may not get notified about that changes and won't sync to the NAS

- Mac1 is set to sleep

- Waking Up Mac2 - It's not getting any new Database because - see above

- Now changing something on Mac2 on that database 

- BTSync is getting notified and syncing to the NAS

- Set Mac2 to sleep

- Now wake up Mac1 again... What will happen... File on NAS is newer (date) (changes from Mac2) - but these changes are not including the not-synced-changes from Mac1.... I don't think BTSync can merge a SQLlite-Database :D so i'm worried, that i will now  lost the older changes (and that wouldn't be solved with an Indexing every 10 Minutes - not in all cases)

 

Is this assumption right?

Share this post


Link to post
Share on other sites

@derBoris Yep, your assumption is correct. Sync is only working on file-level and is definitely not able to merge any data inside files. So, in your worst case you'll lose changes in DB made on Mac1.

For the edge cases. I'm aware of couple of cases for OS X (and actually other POSIX OS) when Sync may miss notifications:

  • there is not enough inotifies in OS (we use inotify to subscribe to changes), though this usually happens more on Linux rather than on OS X
  • some really massive changes (thousands of files) happen in a very short period of time

 

Share this post


Link to post
Share on other sites

Hey @RomanZ,

cool, so i think it should work now. I've set the indexing-Interval to 0 (zero) on both macs and to a very long period (a day) on the NAS.

As long as the OS-X works correct, this should work fine. I think that's enough for my usecase right now :) I will see...

Thanks again for the quick reply

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.