red-lichtie

I deleted the db file and it appears that btsync rescanned the tree and recreated the DB. So now to wait and see if it goes wild again. I'll report what happens either way Thanks rdebath for your help.

Thanks, I tried it and the DB appears to be corrupt. sqlite> pragma integrity_check; *** in database main *** Page 192349: btreeInitPage() returns error code 11 On tree page 257370 cell 30: Child page depth differs Page 194288: btreeInitPage() returns error code 11 Page 195418: btreeInitPage() returns error code 11 ... Page 197207: btreeInitPage() returns error code 11 Page 197223: btreeInitPage() returns error code 11 Page 197208: btreeInitPage() returns error code 11 Page 197266: btreeInitPage() returns error code 11 Page 197281: btreeInitPage() returns error code 11 sqlite> Do I just delete the DB and let a new one be created ?

Completely dies after running for a while (day?). oom-killer kicks in and kills btsync. I tried to circumvent that yesterday my changing the oom parameters for btsync but that lead to my system completely dieing. Here is what was in the syslog the 1st time the oom-likker was called. Jul 22 06:34:11 filesrv kernel: [73164.502786] btsync invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=-17, oom_score_adj=-1000 Jul 22 06:34:11 filesrv kernel: [73164.502791] btsync cpuset=/ mems_allowed=0 Jul 22 06:34:11 filesrv kernel: [73164.502801] Pid: 2755, comm: btsync Not tainted 3.5.0-34-generic #55-Ubuntu Jul 22 06:34:11 filesrv kernel: [73164.502802] Call Trace: Jul 22 06:34:11 filesrv kernel: [73164.502823] [<ffffffff81675e5e>] dump_header+0x86/0x1c0 Jul 22 06:34:11 filesrv kernel: [73164.502832] [<ffffffff812b44d5>] ? security_capable_noaudit+0x15/0x20 Jul 22 06:34:11 filesrv kernel: [73164.502839] [<ffffffff81330bb3>] ? ___ratelimit+0xa3/0x120 Jul 22 06:34:11 filesrv kernel: [73164.502848] [<ffffffff81127103>] oom_kill_process+0x283/0x2b0 Jul 22 06:34:11 filesrv kernel: [73164.502850] [<ffffffff8112759b>] out_of_memory+0x1fb/0x3b0 Jul 22 06:34:11 filesrv kernel: [73164.502853] [<ffffffff8112c806>] __alloc_pages_nodemask+0x776/0x930 Jul 22 06:34:11 filesrv kernel: [73164.502859] [<ffffffff811651d0>] alloc_pages_current+0xb0/0x120 Jul 22 06:34:11 filesrv kernel: [73164.502861] [<ffffffff811235ef>] __page_cache_alloc+0xaf/0xd0 Jul 22 06:34:11 filesrv kernel: [73164.502863] [<ffffffff81125b92>] filemap_fault+0x282/0x450 Jul 22 06:34:11 filesrv kernel: [73164.502877] [<ffffffff8117caef>] ? mem_cgroup_update_page_stat+0x1f/0x60 Jul 22 06:34:11 filesrv kernel: [73164.502881] [<ffffffff811472cf>] __do_fault+0x6f/0x530 Jul 22 06:34:11 filesrv kernel: [73164.502884] [<ffffffff8114a724>] handle_pte_fault+0x94/0x430 Jul 22 06:34:11 filesrv kernel: [73164.502890] [<ffffffff8107a400>] ? lock_hrtimer_base.isra.22+0x30/0x60 Jul 22 06:34:11 filesrv kernel: [73164.502893] [<ffffffff8114b819>] handle_mm_fault+0x259/0x320 Jul 22 06:34:11 filesrv kernel: [73164.502897] [<ffffffff81685d9a>] do_page_fault+0x16a/0x4e0 Jul 22 06:34:11 filesrv kernel: [73164.502902] [<ffffffff8108aa1f>] ? __dequeue_entity+0x2f/0x50 Jul 22 06:34:11 filesrv kernel: [73164.502909] [<ffffffff810125be>] ? __switch_to+0x16e/0x420 Jul 22 06:34:11 filesrv kernel: [73164.502911] [<ffffffff81083eea>] ? finish_task_switch+0x4a/0xf0 Jul 22 06:34:11 filesrv kernel: [73164.502913] [<ffffffff81680adf>] ? __schedule+0x3cf/0x810 Jul 22 06:34:11 filesrv kernel: [73164.502915] [<ffffffff816858b5>] do_async_page_fault+0x35/0x90 Jul 22 06:34:11 filesrv kernel: [73164.502917] [<ffffffff81682915>] async_page_fault+0x25/0x30 Jul 22 06:34:11 filesrv kernel: [73164.502918] Mem-Info: Jul 22 06:34:11 filesrv kernel: [73164.502919] Node 0 DMA per-cpu: Jul 22 06:34:11 filesrv kernel: [73164.502922] CPU 0: hi: 0, btch: 1 usd: 0 Jul 22 06:34:11 filesrv kernel: [73164.502923] CPU 1: hi: 0, btch: 1 usd: 0 Jul 22 06:34:11 filesrv kernel: [73164.502924] CPU 2: hi: 0, btch: 1 usd: 0 Jul 22 06:34:11 filesrv kernel: [73164.502925] CPU 3: hi: 0, btch: 1 usd: 0 Jul 22 06:34:11 filesrv kernel: [73164.502925] Node 0 DMA32 per-cpu: Jul 22 06:34:11 filesrv kernel: [73164.502926] CPU 0: hi: 186, btch: 31 usd: 30 Jul 22 06:34:11 filesrv kernel: [73164.502927] CPU 1: hi: 186, btch: 31 usd: 30 Jul 22 06:34:11 filesrv kernel: [73164.502928] CPU 2: hi: 186, btch: 31 usd: 0 Jul 22 06:34:11 filesrv kernel: [73164.502929] CPU 3: hi: 186, btch: 31 usd: 81 Jul 22 06:34:11 filesrv kernel: [73164.502931] active_anon:732445 inactive_anon:2 isolated_anon:0 Jul 22 06:34:11 filesrv kernel: [73164.502931] active_file:30 inactive_file:0 isolated_file:38 Jul 22 06:34:11 filesrv kernel: [73164.502931] unevictable:0 dirty:0 writeback:0 unstable:0 Jul 22 06:34:11 filesrv kernel: [73164.502931] free:14225 slab_reclaimable:9316 slab_unreclaimable:3475 Jul 22 06:34:11 filesrv kernel: [73164.502931] mapped:51 shmem:388 pagetables:2371 bounce:0 Jul 22 06:34:11 filesrv kernel: [73164.502933] Node 0 DMA free:12248kB min:224kB low:280kB high:336kB active_anon:3548kB inactive_anon:0kB active_file:8kB inactive_file:4kB unevictable:0kB isolated(a non):0kB isolated(file):0kB present:15652kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:4kB slab_unreclaimable:4kB kernel_stack:0kB pagetables:16kB unstable:0kB bounce:0 kB writeback_tmp:0kB pages_scanned:221 all_unreclaimable? yes Jul 22 06:34:11 filesrv kernel: [73164.502936] lowmem_reserve[]: 0 3008 3008 3008 Jul 22 06:34:11 filesrv kernel: [73164.502938] Node 0 DMA32 free:44652kB min:44828kB low:56032kB high:67240kB active_anon:2926232kB inactive_anon:8kB active_file:112kB inactive_file:0kB unevictable:0 kB isolated(anon):0kB isolated(file):152kB present:3080440kB mlocked:0kB dirty:0kB writeback:0kB mapped:204kB shmem:1552kB slab_reclaimable:37260kB slab_unreclaimable:13896kB kernel_stack:1384kB page tables:9468kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:330 all_unreclaimable? yes Jul 22 06:34:11 filesrv kernel: [73164.502941] lowmem_reserve[]: 0 0 0 0 Jul 22 06:34:11 filesrv kernel: [73164.502943] Node 0 DMA: 14*4kB 4*8kB 4*16kB 5*32kB 3*64kB 2*128kB 3*256kB 1*512kB 2*1024kB 2*2048kB 1*4096kB = 12280kB Jul 22 06:34:11 filesrv kernel: [73164.502948] Node 0 DMA32: 457*4kB 480*8kB 356*16kB 184*32kB 106*64kB 44*128kB 29*256kB 5*512kB 1*1024kB 0*2048kB 1*4096kB = 44772kB Jul 22 06:34:11 filesrv kernel: [73164.502953] 504 total pagecache pages Jul 22 06:34:11 filesrv kernel: [73164.502954] 0 pages in swap cache Jul 22 06:34:11 filesrv kernel: [73164.502955] Swap cache stats: add 0, delete 0, find 0/0 Jul 22 06:34:11 filesrv kernel: [73164.502956] Free swap = 0kB Jul 22 06:34:11 filesrv kernel: [73164.502956] Total swap = 0kB Jul 22 06:34:11 filesrv kernel: [73164.514352] 786414 pages RAM Jul 22 06:34:11 filesrv kernel: [73164.514354] 15578 pages reserved Jul 22 06:34:11 filesrv kernel: [73164.514354] 1082 pages shared Jul 22 06:34:11 filesrv kernel: [73164.514355] 755519 pages non-shared Jul 22 06:34:11 filesrv kernel: [73164.514356] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name Jul 22 06:34:11 filesrv kernel: [73164.514372] [ 329] 0 329 4308 46 2 0 0 upstart-udev-br Jul 22 06:34:11 filesrv kernel: [73164.514374] [ 331] 0 331 5466 190 1 -17 -1000 udevd Jul 22 06:34:11 filesrv kernel: [73164.514376] [ 458] 0 458 5465 190 0 -17 -1000 udevd Jul 22 06:34:11 filesrv kernel: [73164.514377] [ 467] 0 467 5422 175 2 -17 -1000 udevd Jul 22 06:34:11 filesrv kernel: [73164.514379] [ 573] 0 573 3797 47 0 0 0 upstart-socket- Jul 22 06:34:11 filesrv kernel: [73164.514381] [ 694] 0 694 2548 577 0 0 0 dhclient Jul 22 06:34:11 filesrv kernel: [73164.514382] [ 856] 0 856 2548 574 1 0 0 dhclient Jul 22 06:34:11 filesrv kernel: [73164.514384] [ 954] 0 954 26455 231 1 0 0 winbindd Jul 22 06:34:11 filesrv kernel: [73164.514385] [ 957] 0 957 30985 282 1 0 0 smbd Jul 22 06:34:11 filesrv kernel: [73164.514398] [ 960] 0 960 12512 148 1 -17 -1000 sshd Jul 22 06:34:11 filesrv kernel: [73164.514402] [ 961] 102 961 5979 94 1 0 0 dbus-daemon Jul 22 06:34:11 filesrv kernel: [73164.514404] [ 962] 0 962 22805 196 3 0 0 nmbd Jul 22 06:34:11 filesrv kernel: [73164.514405] [ 965] 0 965 31114 273 1 0 0 smbd Jul 22 06:34:11 filesrv kernel: [73164.514406] [ 983] 0 983 26273 235 1 0 0 winbindd Jul 22 06:34:11 filesrv kernel: [73164.514410] [ 996] 101 996 62368 79 2 0 0 rsyslogd Jul 22 06:34:11 filesrv kernel: [73164.514414] [ 1070] 0 1070 3946 43 2 0 0 getty Jul 22 06:34:11 filesrv kernel: [73164.514418] [ 1079] 0 1079 3946 41 2 0 0 getty Jul 22 06:34:11 filesrv kernel: [73164.514421] [ 1088] 0 1088 3946 42 3 0 0 getty Jul 22 06:34:11 filesrv kernel: [73164.514425] [ 1091] 0 1091 3946 40 0 0 0 getty Jul 22 06:34:11 filesrv kernel: [73164.514428] [ 1108] 0 1108 3946 41 0 0 0 getty Jul 22 06:34:11 filesrv kernel: [73164.514432] [ 1123] 0 1123 1083 36 3 0 0 acpid Jul 22 06:34:11 filesrv kernel: [73164.514436] [ 1125] 0 1125 4778 59 1 0 0 cron Jul 22 06:34:11 filesrv kernel: [73164.514439] [ 1127] 0 1127 4227 41 1 0 0 atd Jul 22 06:34:11 filesrv kernel: [73164.514443] [ 1129] 103 1129 47434 306 3 0 0 whoopsie Jul 22 06:34:11 filesrv kernel: [73164.514445] [ 1130] 0 1130 5280 58 0 0 0 irqbalance Jul 22 06:34:11 filesrv kernel: [73164.514447] [ 1165] 0 1165 26455 234 1 0 0 winbindd Jul 22 06:34:11 filesrv kernel: [73164.514449] [ 1283] 0 1283 522917 277 1 0 0 console-kit-dae Jul 22 06:34:11 filesrv kernel: [73164.514450] [ 1350] 0 1350 46654 165 1 0 0 polkitd Jul 22 06:34:11 filesrv kernel: [73164.514451] [ 1860] 0 1860 26275 235 1 0 0 winbindd Jul 22 06:34:11 filesrv kernel: [73164.514453] [ 2181] 0 2181 3946 41 3 0 0 getty Jul 22 06:34:11 filesrv kernel: [73164.514454] [ 2750] 1001 2750 826241 709376 2 -17 -1000 btsync Jul 22 06:34:11 filesrv kernel: [73164.514456] [ 4938] 0 4938 26807 212 1 0 0 cron Jul 22 06:34:11 filesrv kernel: [73164.514457] [ 4939] 0 4939 1099 26 3 0 0 sh Jul 22 06:34:11 filesrv kernel: [73164.514459] [ 4940] 0 4940 1075 28 2 0 0 run-parts Jul 22 06:34:11 filesrv kernel: [73164.514460] [ 4943] 0 4943 1099 37 1 0 0 apt Jul 22 06:34:11 filesrv kernel: [73164.514461] [ 5060] 0 5060 31087 17848 1 0 0 update-apt-xapi Jul 22 06:34:11 filesrv kernel: [73164.514463] Out of memory: Kill process 694 (dhclient) score 0 or sacrifice child Jul 22 06:34:11 filesrv kernel: [73164.514509] Killed process 694 (dhclient) total-vm:10192kB, anon-rss:2308kB, file-rss:0kB The system kept on killing processes until it completely died 4 hours later. I've tremoved the OOM adjustment now. The db file is 282M in size with 110728 files in the sync directory taking up 4.4G, with over 800G still free on the drive. When restarting the log fills up with "SyncDb: failed to execute query to save file info" entries [20130622 19:00:18.831] Loading config file version 1.1.42 [20130622 19:00:19.626] Loaded folder /share/data/btsync [20130622 19:00:21.483] SyncDb: failed to execute query to save file info [20130622 19:00:21.483] SyncDb: failed to execute query to save file info [20130622 19:00:21.483] SyncDb: failed to execute query to save file info [20130622 19:02:29.444] SyncDb: failed to execute query to save file info [20130622 19:02:29.446] SyncDb: failed to execute query to save file info [20130622 19:02:29.446] SyncDb: failed to execute query to save file info So as nice as this all promises to be, the memory management needs looking into