Btsync Unexpectedly Crashes On Arm

[Guest glombus]

I have a simple setup.  I have two folders on an x86_64 machine that sync to an ARM machine.  This has worked fine for about a week.

 

Today I upped the ante.  I added a new 40GB folder to sync between machines.  I added the folder on the source machine, the x86_64.  After it indexed, I added it as ro on the ARM machine and the sync began.

 

I stepped away and returned an hour later to find BTSync had died on the ARM machine.

 

I don't see any clear errors in the sync.log file on the ARM machine.  I ran btsync from terminal on the ARM machine to try and get more info.

 

[root@arm .sync]# btsync --nodaemon --config /etc/btsync/btsync.config
total physical memory -1 max disk cache 2097152
Using IP address 192.168.1.5
Loading config file version 1.2.73
UPnP: Could not map UPnP Port on this pass, retrying.
UPnP: Could not map UPnP Port on this pass, retrying.
UPnP: Could not map UPnP Port on this pass, retrying.
UPnP: Unable to map port 192.168.1.5:63806 with UPnP.
Loaded folder /backup/BTSync/folder1
Loaded folder /backup/BTSync/Drop
Killed

 

I didn't send the kill signal.  I don't know why "Killed" shows up, but when it does, it corresponds to the btsync process dying.  There's about a minute between seeing "Loaded folder /backup/BTSync/Drop" and when btsync chokes and the "Killed" message appears.  The 40GB folder never shows up in the stdout.  It seems logical that btsync dies while "Loading" that folder.

Watching top, mem usage never exceeds 18%.  CPU usage is high, but never holds at 100.  Hovering between 80 and upper 90s.

BitTorrent Sync 1.2.73 on the ARM machine

 

Based on the churning I hear from the USB hard drive on the ARM machine, it sounds like it's dying while indexing this new 40GB folder.

Worst of all is that since the sync folders don't load in the WebUI prior to the crash, I can't go into the WebUI on the ARM machine and remove this folder that seems to be causing the issue.

[Guest glombus]

Seems like nfsd was killing btsync.  I don't know how the oom (out of memory) kill works, but it seems like nfsd sent up a red flag and btsync was chosen at the process to get killed.  Looking at top I really don't see the mem usage for btsync going that high so I don't understand this.  I didn't dig into this too deep, but disabling nfsd seemed to resolve this issue (at least, in the first few hours it was disabled)

[  680.266512] nfsd invoked oom-killer: gfp_mask=0xd0, order=0, oom_score_adj=0[  680.266553] nfsd cpuset=/ mems_allowed=0[  680.266574] CPU: 0 PID: 304 Comm: nfsd Not tainted 3.10.19-3-ARCH #1[  680.266643] [<c0012d88>] (unwind_backtrace+0x0/0xf0) from [<c0010c88>] (show_stack+0x10/0x14)[  680.266696] [<c0010c88>] (show_stack+0x10/0x14) from [<c059a7dc>] (dump_header.isra.15+0x78/0x1b0)[  680.266735] [<c059a7dc>] (dump_header.isra.15+0x78/0x1b0) from [<c00b3450>] (oom_kill_process+0x280/0x3f8)[  680.266764] [<c00b3450>] (oom_kill_process+0x280/0x3f8) from [<c00b3a18>] (out_of_memory+0x268/0x2b4)[  680.266799] [<c00b3a18>] (out_of_memory+0x268/0x2b4) from [<c00b7a78>] (__alloc_pages_nodemask+0x8fc/0x934)[  680.266837] [<c00b7a78>] (__alloc_pages_nodemask+0x8fc/0x934) from [<c058b860>] (svc_alloc_arg+0x7c/0x178)[  680.266867] [<c058b860>] (svc_alloc_arg+0x7c/0x178) from [<c058bcb0>] (svc_recv+0x38/0x3dc)[  680.267120] [<c058bcb0>] (svc_recv+0x38/0x3dc) from [<bf0cc758>] (nfsd+0x9c/0x120 [nfsd])[  680.267287] [<bf0cc758>] (nfsd+0x9c/0x120 [nfsd]) from [<c0043b18>] (kthread+0xa4/0xb0)[  680.267325] [<c0043b18>] (kthread+0xa4/0xb0) from [<c000dc18>] (ret_from_fork+0x14/0x3c)[  680.267338] Mem-info:[  680.267350] Normal per-cpu:[  680.267362] CPU    0: hi:   42, btch:   7 usd:   7[  680.267389] active_anon:11337 inactive_anon:39 isolated_anon:0 active_file:906 inactive_file:2069 isolated_file:0 unevictable:0 dirty:831 writeback:0 unstable:0 free:34780 slab_reclaimable:1309 slab_unreclaimable:992 mapped:216 shmem:75 pagetables:226 bounce:0 free_cma:26602[  680.267457] Normal free:139120kB min:32768kB low:40960kB high:49152kB active_anon:45348kB inactive_anon:156kB active_file:3624kB inactive_file:8276kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:225280kB managed:102680kB mlocked:0kB dirty:3324kB writeback:0kB mapped:864kB shmem:300kB slab_reclaimable:5236kB slab_unreclaimable:3968kB kernel_stack:664kB pagetables:904kB unstable:0kB bounce:0kB free_cma:106408kB writeback_tmp:0kB pages_scanned:10499 all_unreclaimable? yes[  680.267473] lowmem_reserve[]: 0 0[  680.267491] Normal: 14*4kB (UE) 5*8kB (UMC) 5*16kB (UE) 6*32kB (UEC) 10*64kB (UEM) 5*128kB (UEMC) 5*256kB (UMC) 2*512kB (EC) 2*1024kB (MC) 3*2048kB (EMC) 31*4096kB (UMRC) = 139120kB[  680.267574] 3050 total pagecache pages[  680.267588] 0 pages in swap cache[  680.267599] Swap cache stats: add 0, delete 0, find 0/0[  680.267609] Free swap  = 0kB[  680.267617] Total swap = 0kB[  680.296409] 56320 pages of RAM[  680.296467] 34925 free pages[  680.296479] 2943 reserved pages[  680.296488] 1662 slab pages[  680.296497] 265531 pages shared[  680.296505] 0 pages swap cached[  680.296516] [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name[  680.296565] [   58]     0    58    67171       75      75        0             0 systemd-journal[  680.296587] [   91]     0    91     2349       80       7        0         -1000 systemd-udevd[  680.296607] [  124]     0   124      694      132       4        0             0 crond[  680.296626] [  126]     0   126      742       48       5        0             0 systemd-logind[  680.296644] [  128]    81   128      676       54       6        0          -900 dbus-daemon[  680.296665] [  131]     0   131      440       19       4        0             0 agetty[  680.296683] [  132]     0   132      440       18       5        0             0 agetty[  680.296701] [  262]     0   262      504       45       5        0             0 dhcpcd[  680.296720] [  263]     0   263     1584       92       6        0         -1000 sshd[  680.296738] [  267]    87   267     1251      105       6        0             0 ntpd[  680.296757] [  274]    32   274      565       50       5        0             0 rpcbind[  680.296782] [  311]     0   311      627       32       5        0             0 rpc.idmapd[  680.296803] [  324]     0   324      846      186       5        0             0 rpc.mountd[  680.296821] [  325]     0   325     2566      142       8        0             0 sshd[  680.296839] [  329]  1001   329      948       66       5        0             0 systemd[  680.296857] [  330]  1001   330     2598      146       8        0             0 sshd[  680.296875] [  331]  1001   331     1282      171       5        0             0 (sd-pam)[  680.296894] [  332]  1001   332      821       83       5        0             0 bash[  680.296914] [  334]  1001   334      716       39       5        0             0 su[  680.296932] [  335]     0   335      823       86       5        0             0 bash[  680.296950] [  357]     0   357    25137     9905      42        0             0 btsync[  680.296965] Out of memory: Kill process 357 (btsync) score 156 or sacrifice child[  680.316766] Killed process 357 (btsync) total-vm:100548kB, anon-rss:38864kB, file-rss:756kB

[brett]

I have a 1TB external drive that syncs nicely between two windows machines, however, I have a similar issue with sync crashing on ARM (a raspberry pi) and two possible causes.

1. The permissions are not set to allow sync to write to the folder(s)
2. The index file that sync creates is too bulky for the pi to process with its limited memory and it falls over when syncing large directories

For information, I wouldn't recommend using sync to do the initial file transfer. Use rsync on linux, robocopy or FreeFileSync on Windows. My guess is that the crash in my case is a result of poor permissions.

[freigeist]

Did you get more stable on the pi now? I still experience unstable performance on my pi.

[Guest glombus]

Did you get more stable on the pi now? I still experience unstable performance on my pi.

 

I recently tried again on my Pi.  I am using the latest BTSync binary and the latest raspbian image and I'm only using 16MB for the GPU.  It's a headless setup.  It seems to be running fine and has not crashed since I set it up last week.

 

Keep in mind, I'm not running nfsd this time around or anything else besides BTSync.