ExHealer

[Solved] too many nodes to send

Recommended Posts

Hello!

I have two freebsd 64 storages, one master (RW), second slave (RO).

Today i upgrade my freebsd storages to 2.3.7 and folders stop syncing.

On master:

[20160616 05:30:46.617] MC[6064] [11ED]: processing get_nodes message for /uploads
[20160616 05:30:46.795] SF[6064] [11ED]: Received request "get_nodes"
[20160616 05:30:46.796] MC[6064] [11ED]: processing get_nodes message for /uploads/00
[20160616 05:30:46.804] MC[6064] [11ED]: processing get_nodes message for /uploads/01
[20160616 05:30:46.815] MC[6064] [11ED]: processing get_nodes message for /uploads/02
[20160616 05:30:46.826] MC[6064] [11ED]: processing get_nodes message for /uploads/03
[20160616 05:30:46.836] MC[6064] [11ED]: processing get_nodes message for /uploads/04
[20160616 05:30:46.847] MC[6064] [11ED]: processing get_nodes message for /uploads/05
[20160616 05:30:46.858] MC[6064] [11ED]: processing get_nodes message for /uploads/06
[20160616 05:30:46.869] MC[6064] [11ED]: processing get_nodes message for /uploads/07
[20160616 05:30:46.880] MC[6064] [11ED]: processing get_nodes message for /uploads/08
[20160616 05:30:46.891] MC[6064] [11ED]: processing get_nodes message for /uploads/09
[20160616 05:30:46.899] MC[6064] [11ED]: processing get_nodes message for /uploads/0a
[20160616 05:30:46.910] MC[6064] [11ED]: processing get_nodes message for /uploads/0b
[20160616 05:30:46.921] MC[6064] [11ED]: processing get_nodes message for /uploads/0c
[20160616 05:30:46.932] MC[6064] [11ED]: processing get_nodes message for /uploads/0d
[20160616 05:30:46.942] MC[6064] [11ED]: processing get_nodes message for /uploads/0e
[20160616 05:30:46.953] MC[6064] [11ED]: too many nodes to send 100596, stopping at /uploads/0e
[20160616 05:30:46.953] MC[6064] [11ED]: processing get_nodes message for /uploads/0f
[20160616 05:30:46.964] MC[6064] [11ED]: too many nodes to send 107513, stopping at /uploads/0f
[20160616 05:30:46.964] MC[6064] [11ED]: processing get_nodes message for /uploads/10
...
[20160616 05:30:49.495] MC[6064] [11ED]: processing get_nodes message for /uploads/ff
[20160616 05:30:49.505] MC[6064] [11ED]: too many nodes to send 1752998, stopping at /uploads/ff

 

On slave:

[20160616 05:37:02.240] MC[6064] [F8A9]: requesting nodes for root
[20160616 05:37:02.373] SF[6064] [F8A9]: Received request "nodes"
[20160616 05:37:02.373] MC[6064] [F8A9]: processing nodes message for /
[20160616 05:37:02.373] MC[6064] [F8A9]: will request nodes for /uploads
[20160616 05:37:02.373] MC[6064] [F8A9]: sending get_nodes message
[20160616 05:37:02.600] SF[6064] [F8A9]: Received request "nodes"
[20160616 05:37:02.600] MC[6064] [F8A9]: processing nodes message for /uploads
[20160616 05:37:02.600] MC[6064] [F8A9]: will request files for /uploads
[20160616 05:37:02.600] MC[6064] [F8A9]: will request nodes for /uploads/00
[20160616 05:37:02.600] MC[6064] [F8A9]: will request nodes for /uploads/01
[20160616 05:37:02.600] MC[6064] [F8A9]: will request nodes for /uploads/02
[20160616 05:37:02.600] MC[6064] [F8A9]: will request nodes for /uploads/03
[20160616 05:37:02.600] MC[6064] [F8A9]: will request nodes for /uploads/04
...
[20160616 05:36:15.572] MC[6064] [F8A9]: will request nodes for /uploads/f7
[20160616 05:36:15.572] MC[6064] [F8A9]: will request nodes for /uploads/f8
[20160616 05:36:15.572] MC[6064] [F8A9]: will request nodes for /uploads/f9
[20160616 05:36:15.572] MC[6064] [F8A9]: will request nodes for /uploads/fa
[20160616 05:36:15.572] MC[6064] [F8A9]: will request nodes for /uploads/fb
[20160616 05:36:15.572] MC[6064] [F8A9]: will request nodes for /uploads/fc
[20160616 05:36:15.572] MC[6064] [F8A9]: will request nodes for /uploads/fd
[20160616 05:36:15.572] MC[6064] [F8A9]: will request nodes for /uploads/fe
[20160616 05:36:15.572] MC[6064] [F8A9]: will request nodes for /uploads/ff
[20160616 05:36:15.572] MC[6064] [F8A9]: sending get_nodes message

 

And same messages repeate again and again.

 

On slave every time i see:

[20160616 05:17:16.483] MC[6064] [F8A9]: will request files for /uploads/00
[20160616 05:17:16.483] MC[6064] [F8A9]: will request files for /uploads/00/0079f2bfd9aa28481db51b74a9f5c61c728775*
[20160616 05:17:16.483] MC[6064] [F8A9]: will request files for /uploads/00/0079f2bfd9aa28481db51b74a9f5c61c728775_lq*
[20160616 05:17:16.483] MC[6064] [F8A9]: will request files for /uploads/00/0079f2bfd9aa28481db51b74a9f5c61c728775_mq*
...
[20160616 05:37:01.702] MC[6064] [F8A9]: will request files for /uploads/ff/f9091f803e977fb16da90dc8e6d241bd3cc351*
[20160616 05:37:01.702] MC[6064] [F8A9]: will request files for /uploads/ff/f9091f803e977fb16da90dc8e6d241bd3cc351_lq*
[20160616 05:37:01.702] MC[6064] [F8A9]: will request files for /uploads/ff/f9091f803e977fb16da90dc8e6d241bd3cc351_mq*

 

Sync does not start.

 

Whats wrong?

 

Share this post


Link to post
Share on other sites

You've most likely hit the packet size limit. you have 100,000 files in one subfolder? 

in Sync on RW go to prefs -> advanced -> power user setting and double "max packet size" parameter. By default it's 32. Or better check the log on RO peer for line "ERROR: packet size is too big" and put "max packet size" bigger than indicated in that line. 

Share this post


Link to post
Share on other sites

Already done.

I'm using this configs.

slave.conf
 

{
  "device_name": "slave",
  "listening_port" : 13201,
  "pid_file" : "/home/.btsync/var/run/user.pid",
  "storage_path" : "/home/user/.sync",
  "use_upnp" : false,
  "folder_rescan_interval" : 600,
  "download_limit" : 0,
  "upload_limit" : 0,
  "recv_buf_size" : 128,
  "send_buf_size" : 128,
  "rate_limit_local_peers" : false,
  "lan_encrypt_data" : false,
  "disk_low_priority" : false,
  "max_packet_size" : 2048,
  "shared_folders" :
  [
    {
      "secret" : "SLAVE_SECRET_HERE",
      "dir" : "/home/user/sync",
      "use_relay_server" : false,
      "use_tracker" : false,
      "search_lan" : false,
      "use_sync_trash" : false,
      "known_hosts" :
      [
        "master:13201"
      ]
    }
  ]
}

 

master.conf

{
  "device_name": "master",
  "listening_port" : 13201,
  "pid_file" : "/home/.btsync/var/run/user.pid",
  "storage_path" : "/home/user/.sync",
  "use_upnp" : false,
  "folder_rescan_interval" : 600,
  "download_limit" : 0,
  "upload_limit" : 0,
  "recv_buf_size" : 128,
  "send_buf_size" : 128,
  "rate_limit_local_peers" : false,
  "lan_encrypt_data" : false,
  "disk_low_priority" : false,
  "bind_interface" : "lagg0.101",
  "max_packet_size" : 2048,
  "shared_folders" :
  [
    {
      "secret" : "MASTER_SECRET_HERE",
      "dir" : "/home/user/sync",
      "use_relay_server" : false,
      "use_tracker" : false,
      "search_lan" : false,
      "use_sync_trash" : false,
      "known_hosts" :
      [
        "slave:13201"
      ]
    }
  ]
}

 

What can be wrong?

 

Share this post


Link to post
Share on other sites

 you have 100,000 files in one subfolder? 

what is reported in ERROR: packet size is too big? 

To tell what is wrong we need full logs from both machines. otherwise those will be just assumptions. 

Share this post


Link to post
Share on other sites

I have 269 subfolders in /sync/upload folder, and from 5k to 10k files in each subfolder.

Total from 1 345 000 to 2 690 000 files in 269 folders.


Where can I send log files?

Share this post


Link to post
Share on other sites

Hi,

 

have you solved the problem?

I got the same...

Too many nodes to send and huge numbers at RO :

SyncSocket[0x0000023c01ea0960]: ERROR: packet size is too big, size = 63391175

 

Regards,

Jaroslav

Share this post


Link to post
Share on other sites

yes, that was the question, thanks. Sorry, don' have builds with the fix for Windows now, so I guess you will need to wait for the official release...

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.