johan.mach

"got Metadata With Too Large Size" Then Crash

Recommended Posts

Hi, 

 

I have searched and searched but haven't come across anyone with this issue before.

 

I am syncing some large files (up to 2.5TB big, they are virtual machine backups) and regularly I get the following in my log followed 10-15 seconds by btsync crashing:

 [20140120 09:56:58.551] Got metadata with too large size 13522946

The size is not always the same, and sometimes I'll get a few identical messages followed by the crash - the btsync process simply disappears.

 

I've turned on debug logging, but that gives absolutely no more information, and the crash is just as sudden.

 

Constantly restarting btsync shows it will sometimes sync a bit of data for a few different torrents but eventually it will get around to whatever file has this metadata and then crash.

 

btsync on both peers is running on FreeBSD 9.1 x64 (it's a FreeNAS box with some additions). I have plenty of other sync folders on other boxes in exactly the same environment who don't have this issue.

 

Any ideas?

 

Cheers,

Johan

 

 

Share this post


Link to post
Share on other sites

Hi,

 

I would appreciate if you can share full debug logs from both machines as well as crash dump of crashed process (should be in .sync subfolder in the same place where btsync binary stored).

 

As soon as dumps and logs get analysed - I'll share results / recommendations.

Share this post


Link to post
Share on other sites

Hi Roman,

 

I have emailed syncapp@bittorrent.com with the log files as requested. 

 

I censored the logs a bit to protect internal details, but essentially it comes down to a read only peer syncing from a full access peer, and the process I followed was:

  •   Start the full-access-peer btsync process
  •   Start the read-only-peer btsync process
  •   Read only peer ran for a couple of minutes before the btsync process disappeared
  •   Stopped the full-access-peer btsync process
It's crashing with a segfault, per:
 
Jan 21 08:26:45 read-only-peer kernel: pid 80758 (btsync), uid 0: exited on signal 11Jan 21 08:29:05 read-only-peer kernel: pid 80868 (btsync), uid 0: exited on signal 11Jan 21 08:31:48 read-only-peer kernel: pid 81017 (btsync), uid 0: exited on signal 11Jan 21 08:34:00 read-only-peer kernel: pid 81127 (btsync), uid 0: exited on signal 11Jan 21 08:36:31 read-only-peer kernel: pid 81226 (btsync), uid 0: exited on signal 11...

I've managed to get a coredump (needed to enable with "sysctl kern.coredump=1" on FreeNAS) and I'm emailing it to the ticket now.

 

Thanks,

Johan

Edited by johan.mach

Share this post


Link to post
Share on other sites

Hi Roman,

 

That's really bad news - I thought btsync didn't have any particular limitations (at least nothing is mentioned in the FAQs etc).

 

The interesting thing is the file that it is choking on is about 1.2TB, but I have successfully synced files up to 2.2TB before.

 

Any idea when this issue will be resolved?

 

Thanks

Johan

Share this post


Link to post
Share on other sites

Johan,

 

This limitation is more a bug, than limitation. We want to fix it as technically there should be no limit for file size (well, unless there is some filesystem which supports it).

 

The product is in Beta phase now, so I cannot claim for any ETA now, sorry.

Share this post


Link to post
Share on other sites

Hey there this following question is completely unrelated to Sync but is related to the bit torrent protocol and the metadata_size param in the extended handshake....

 

[The rest of this post has been removed]

Edited by GreatMarko
This forum is for discussing problems relating to BitTorrent Sync. Questions relating to torrents themselves (or the protocol behind them) are off-topic

Share this post


Link to post
Share on other sites

Johan,

 

This limitation is more a bug, than limitation. We want to fix it as technically there should be no limit for file size (well, unless there is some filesystem which supports it).

 

The product is in Beta phase now, so I cannot claim for any ETA now, sorry.

 

I am also experiencing this problem.  I helped a friend recover a dying hard drive and this is preventing me from backing up the image, it's about 1.4TB.  If it matters, my backup server is using encryption keys, not RW or RO. 

 

[20140811 06:59:45.224] Loading config file version 1.3.106
[20140811 06:59:45.259] Loaded folder /btsync/data/deaaf59d
[20140811 07:00:53.245] Blocked downloading file VOPXHNXLQAMHSZZZHOJJXHHC7AJRMVMVHPQIFTY due Timeout due to inactivity
[20140811 07:01:55.725] Got metadata with too large size 27465664
 

Can you take a look at the problem again and see if you can fix it soon?

 

 

Thanks,

 

Eric

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.