hawkman

Synced folder size / number of files limit?

Recommended Posts

Is there a limit to the number of files or the total file size for a synced folder?

I have a folder with 1.18 TB and 966 files that I just set up for syncing. I haven't actually set up the target yet because it took almost a day to index. However once it completed the indexing, it seemed to stop at 1.0 TB and 911 files.

Is this a limit of the program? Or am I doing something wrong?

Also, why does it take so long to index?

Share this post


Link to post
Share on other sites

Sync creates an index of blocks in the files. not just the files. The blocks are in 4MB chunks, so I can see why it would take awhile to index. It took me about 14 hours to index 500GB on my little raspberry pi.

In regards to your first question, there is a limit on the amount of files, but you are nowhere close (It's like 2M or something). Are you including the hidden files and folders that are generated with Sync? I'm pretty sure those aren't indexed, but it would show up in the folder contents.

Share this post


Link to post
Share on other sites

I actually believe the current limit is 200,000 files (See the "Unofficial FAQ")... still no where near the OP's ~1000 files! :)

According to kos, it's 1M. Where did the 200k number come from? I had to look it up, I knew I read somewhere that it wasn't in the thousands... There's so much info floating around here, I'm glad you're taking on getting the unofficial FAQ up. That will help a lot!

http://forum.bittorr...ion/#entry42312

Share this post


Link to post
Share on other sites

According to kos, it's 1M. Where did the 200k number come from?

You're absolutely right! - Unoff. FAQ has been updated accordingly!

Originally, the limit was 100,000, then it got increased to 200,000 - I obviously missed the subsequent update to 1M - so thanks for the tip! :)

Share this post


Link to post
Share on other sites

OK now I'm confused.

Here's what I see from the command line:

- "du -h" gives "1.1T"

- "find . -type f | wc -l" gives 942

From Mac OS X, the folder info shows "1.18 TB on disk for 966 items"

From Bittorrent Sync, I see "1.0 TB in 911 files"

I'm not sure what's right or wrong here...?

Share this post


Link to post
Share on other sites

OK now I have worked out the unix command to match the Mac OS X Finder file count:

%find . \! -name ".*" | wc -l

966

So now why is the Sync file count / file size different? Is there a bug here?

<edit>

Also, why is the size so precisely shown as 1.0TB?

Share this post


Link to post
Share on other sites

Can you create an md5 checksum index of the store that shows the most files, and then do a check against the other stores? That will tell you everything you need to know about the consistency ...

Share this post


Link to post
Share on other sites

Actually I haven't yet synced. I was just trying to reconcile what the UI says versus what's on the disk.

I'm starting the sync now on the LAN. At 2.5 MB/s this could take a while...

Share this post


Link to post
Share on other sites

Is there an actual file limit?

I have one folder set up with this:

238.9 GB in 2403509 files.

It takes a VERY long time to index and it's process on a Haswell linux machines is almost always at 100%, but it still indexed 239GB in 2.4 million files. I don't recommend it and I really hope that the developers can improve on this though.

Here's what the .sync folder looks like:

total 2.2G

-rw-------. 1 root root 69K Jul 29 10:00 xxxxx.dmp

-rw-r--r--. 1 root root 2.1G Aug 1 10:56 xxxxx.db

-rw-r--r--. 1 root root 224K Aug 1 10:57 xxxxx.db-shm

-rw-r--r--. 1 root root 16M Aug 1 10:57 xxxxx.db-wal

-rw-r--r--. 1 root root 2.8K Aug 1 11:00 settings.dat

-rw-r--r--. 1 root root 2.8K Aug 1 10:30 settings.dat.old

-rw-r--r--. 1 root root 773 Aug 1 10:57 sync.dat

-rw-r--r--. 1 root root 773 Aug 1 10:47 sync.dat.old

-rw-r--r--. 1 root root 62M Aug 1 11:05 sync.log

-rw-r--r--. 1 root root 5 Jul 31 10:17 sync.pid

-rw-r--r--. 1 root root 90K Jul 31 10:17 webui.zip

Share this post


Link to post
Share on other sites

Is there an actual file limit?

*sigh* I don't know how many more times we need to say this - No, there are no limits imposed on the number/size of files you can Sync - the only limitation is your actual hardware.

Please read "Is there a limit on the number/size of files I can sync?" in the Unofficial FAQ

Share this post


Link to post
Share on other sites

i have 5.5 Tb and 650k files and BTS only shows 2.6 Tb and 110k files....
it has been online for a few weeks and now it's just there.. no cpu activity...
help?

 

 

(i tryed posting the screen caps but it's not allowed?

"You are not allowed to use that image extension on this community.")

Edited by jpcqa

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.