[[Solved] BTSync randomly deletes files]

@Helen, Upgrading to the latest btsync version fixed the issue. I'm happy as clam! (this is regarding my issue at the top of this thread)

[[Solved] BTSync randomly deletes files]

Setup

I have the latest BTSync running on two headless-linux devices.

The two devices are linked; Device A is located in my home while device B is a virtual server in some datacenter.

Device A generates approx. 50 images (100Kb each) on a regular basis (45 mins on average) and stores those under a new sub-directory; At the same time, it creates a static symbolic link which points to the new images directory just created.

Issue Observed

All files are synced from device A to B as expected. However very often, the files that were added are unexpectedly removed from device A and device B. This typically occurs anywhere from 30 to 90 minutes after the files completed syncing from device A to B.

Question

I'm looking for suggestions on how to trace which device is triggering the deletion of the files and for what reason?

[Linux Server - Out Of Memory (version 1.1.42)]

I have a similar case where btsync is killed because of oom - see segment below from /var/log/messages.

The *.db file is 631Mb in size and it is NOT corrupt (I checked using @rdebath sqlite3 test above).

 

Any ideas what other checks and test I can do to identify the issue?

 

My setup is a Raspberry Pi, Synced 49GB in 307418 files, and I don't seem to have an issue disk space issues.

 

Thanks

Nov  4 16:47:02 sync kernel: [ 3500.806921] btsync invoked oom-killer: gfp_mask=0x200da, order=0, oom_score_adj=0Nov  4 16:47:02 sync kernel: [ 3500.806960] CPU: 0 PID: 2199 Comm: btsync Not tainted 3.12.22+ #691Nov  4 16:47:02 sync kernel: [ 3500.807027] [<c0013ec0>] (unwind_backtrace+0x0/0xf0) from [<c0011284>] (show_stack+0x10/0x14)Nov  4 16:47:02 sync kernel: [ 3500.807093] [<c0011284>] (show_stack+0x10/0x14) from [<c041d794>] (dump_header.isra.13+0x74/0x1b0)Nov  4 16:47:02 sync kernel: [ 3500.807134] [<c041d794>] (dump_header.isra.13+0x74/0x1b0) from [<c009d900>] (oom_kill_process+0x2dc/0x454)Nov  4 16:47:02 sync kernel: [ 3500.807162] [<c009d900>] (oom_kill_process+0x2dc/0x454) from [<c009df90>] (out_of_memory+0x2e0/0x344)Nov  4 16:47:02 sync kernel: [ 3500.807195] [<c009df90>] (out_of_memory+0x2e0/0x344) from [<c00a1de4>] (__alloc_pages_nodemask+0x89c/0x8e0)Nov  4 16:47:02 sync kernel: [ 3500.807223] [<c00a1de4>] (__alloc_pages_nodemask+0x89c/0x8e0) from [<c00c9c7c>] (read_swap_cache_async+0x154/0x22c)Nov  4 16:47:02 sync kernel: [ 3500.807249] [<c00c9c7c>] (read_swap_cache_async+0x154/0x22c) from [<c00c9db8>] (swapin_readahead+0x64/0xa8)Nov  4 16:47:02 sync kernel: [ 3500.807289] [<c00c9db8>] (swapin_readahead+0x64/0xa8) from [<c00bb5a4>] (handle_mm_fault+0x6b8/0x9bc)Nov  4 16:47:02 sync kernel: [ 3500.807337] [<c00bb5a4>] (handle_mm_fault+0x6b8/0x9bc) from [<c04253e8>] (do_page_fault+0x240/0x3f0)Nov  4 16:47:02 sync kernel: [ 3500.807368] [<c04253e8>] (do_page_fault+0x240/0x3f0) from [<c000835c>] (do_DataAbort+0x34/0x98)Nov  4 16:47:02 sync kernel: [ 3500.807394] [<c000835c>] (do_DataAbort+0x34/0x98) from [<c0423dfc>] (__dabt_usr+0x3c/0x40)Nov  4 16:47:02 sync kernel: [ 3500.807407] Exception stack(0xdb327fb0 to 0xdb327ff8)Nov  4 16:47:02 sync kernel: [ 3500.807427] 7fa0:                                     aaacd158 008921a8 00000000 00891fc8Nov  4 16:47:02 sync kernel: [ 3500.807448] 7fc0: a4d0abdc 00479c24 a4d0abd8 00000000 00479c20 a4d0acdc 00000000 003f2280Nov  4 16:47:02 sync kernel: [ 3500.807480] 7fe0: 08b77fa0 a4d0abd0 aaacd108 000aaf04 20000010 ffffffffNov  4 16:47:02 sync kernel: [ 3500.807492] Mem-info:Nov  4 16:47:02 sync kernel: [ 3500.807503] Normal per-cpu:Nov  4 16:47:02 sync kernel: [ 3500.807515] CPU    0: hi:  186, btch:  31 usd:  31Nov  4 16:47:02 sync kernel: [ 3500.807544] active_anon:50411 inactive_anon:50461 isolated_anon:32Nov  4 16:47:02 sync kernel: [ 3500.807544]  active_file:104 inactive_file:65 isolated_file:0Nov  4 16:47:02 sync kernel: [ 3500.807544]  unevictable:0 dirty:0 writeback:14 unstable:0Nov  4 16:47:02 sync kernel: [ 3500.807544]  free:2041 slab_reclaimable:5901 slab_unreclaimable:895Nov  4 16:47:02 sync kernel: [ 3500.807544]  mapped:112 shmem:0 pagetables:390 bounce:0Nov  4 16:47:02 sync kernel: [ 3500.807544]  free_cma:0Nov  4 16:47:02 sync kernel: [ 3500.807609] Normal free:8164kB min:8192kB low:10240kB high:12288kB active_anon:201644kB inactive_anon:201844kB active_file:416kB inactive_file:260kB unevictable:0kB isolated(anon):128kB isolated(file):0kB present:458752kB managed:447996kB mlocked:0kB dirty:0kB writeback:56kB mapped:448kB shmem:0kB slab_reclaimable:23604kB slab_unreclaimable:3580kB kernel_stack:632kB pagetables:1560kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:1272 all_unreclaimable? yesNov  4 16:47:02 sync kernel: [ 3500.807624] lowmem_reserve[]: 0 0Nov  4 16:47:02 sync kernel: [ 3500.807640] Normal: 1109*4kB (UE) 216*8kB (R) 53*16kB (R) 6*32kB (R) 1*64kB (R) 1*128kB (R) 1*256kB (R) 1*512kB (R) 0*1024kB 0*2048kB 0*4096kB = 8164kBNov  4 16:47:02 sync kernel: [ 3500.807719] 9902 total pagecache pagesNov  4 16:47:02 sync kernel: [ 3500.807734] 9733 pages in swap cacheNov  4 16:47:02 sync kernel: [ 3500.807745] Swap cache stats: add 122897, delete 113164, find 41446/49296Nov  4 16:47:02 sync kernel: [ 3500.807753] Free swap  = 0kBNov  4 16:47:02 sync kernel: [ 3500.807760] Total swap = 102396kBNov  4 16:47:02 sync kernel: [ 3500.884014] 114688 pages of RAMNov  4 16:47:02 sync kernel: [ 3500.884040] 2266 free pagesNov  4 16:47:02 sync kernel: [ 3500.884052] 2689 reserved pagesNov  4 16:47:02 sync kernel: [ 3500.884061] 3719 slab pagesNov  4 16:47:02 sync kernel: [ 3500.884069] 169 pages sharedNov  4 16:47:02 sync kernel: [ 3500.884076] 9738 pages swap cachedNov  4 16:47:02 sync kernel: [ 3500.884087] [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj nameNov  4 16:47:02 sync kernel: [ 3500.884147] [  158]     0   158      722        0       5      133         -1000 udevdNov  4 16:47:02 sync kernel: [ 3500.884172] [  299]     0   299      721        0       5      137         -1000 udevdNov  4 16:47:02 sync kernel: [ 3500.884191] [  305]     0   305      721        0       5      137         -1000 udevdNov  4 16:47:02 sync kernel: [ 3500.884210] [ 1608]     0  1608      438        6       4       18         -1000 ifplugdNov  4 16:47:02 sync kernel: [ 3500.884228] [ 1637]     0  1637      438        6       4       16         -1000 ifplugdNov  4 16:47:02 sync kernel: [ 3500.884245] [ 1844]     0  1844     6994       33       7       86             0 rsyslogdNov  4 16:47:02 sync kernel: [ 3500.884262] [ 1893]     0  1893     1225       40       5      390         -1000 dhclientNov  4 16:47:02 sync kernel: [ 3500.884280] [ 1908]     0  1908   130932    91041     237    23316             0 btsyncNov  4 16:47:02 sync kernel: [ 3500.884299] [ 1950]     0  1950      950       26       5       31             0 cronNov  4 16:47:02 sync kernel: [ 3500.884331] [ 1983]   104  1983      795        0       5       58             0 dbus-daemonNov  4 16:47:02 sync kernel: [ 3500.884351] [ 2023]   107  2023      875        4       4       67             0 avahi-daemonNov  4 16:47:02 sync kernel: [ 3500.884369] [ 2024]   107  2024      846        0       4       56             0 avahi-daemonNov  4 16:47:02 sync kernel: [ 3500.884388] [ 2067]     0  2067     1184        0       5       62             0 cnid_metadNov  4 16:47:02 sync kernel: [ 3500.884405] [ 2072]     0  2072     3607        0       6      116             0 afpdNov  4 16:47:02 sync kernel: [ 3500.884425] [ 2077] 65534  2077      557        0       5       39             0 noip2Nov  4 16:47:02 sync kernel: [ 3500.884442] [ 2090]   102  2090     1389       49       6       51             0 ntpdNov  4 16:47:02 sync kernel: [ 3500.884459] [ 2126]     0  2126     1554        0       5      106         -1000 sshdNov  4 16:47:02 sync kernel: [ 3500.884478] [ 2158] 65534  2158      505        4       5       26             0 thdNov  4 16:47:02 sync kernel: [ 3500.884495] [ 2169]     0  2169      937        0       5       32             0 gettyNov  4 16:47:02 sync kernel: [ 3500.884513] [ 2170]     0  2170      937        0       5       32             0 gettyNov  4 16:47:02 sync kernel: [ 3500.884545] [ 2171]     0  2171      937        0       5       32             0 gettyNov  4 16:47:02 sync kernel: [ 3500.884565] [ 2172]     0  2172      937        0       5       32             0 gettyNov  4 16:47:02 sync kernel: [ 3500.884583] [ 2173]     0  2173      937        0       6       32             0 gettyNov  4 16:47:02 sync kernel: [ 3500.884601] [ 2174]     0  2174      937        0       6       32             0 gettyNov  4 16:47:02 sync kernel: [ 3500.884618] [ 2175]     0  2175      517        0       5       31             0 gettyNov  4 16:47:02 sync kernel: [ 3500.884635] [ 2177]     0  2177     2400        0       7      152             0 sshdNov  4 16:47:02 sync kernel: [ 3500.884653] [ 2181]  1000  2181     2400       25       7      129             0 sshdNov  4 16:47:02 sync kernel: [ 3500.884672] [ 2182]  1000  2182     1561        0       7      463             0 bashNov  4 16:47:02 sync kernel: [ 3500.884689] [ 2196]  1000  2196      847        8       5       12             0 tail

[Folder_Rescan_Interval Can Be Disabled On Read-Only Device?]

BTsync is drawing quite a lot of CPU resources from my Raspberry Pi (currently I have approx. 97k files that total 15GB and I expect that to double before it plateaus). 

 

Based on other articles in this forum, i'm considering to increase the folder_rescan_interval to be a much higher number. Here comes my question ... 

 

The Raspberry Pi simply serves as a Read-Only device, in other words, no files will be changing on this device, it only servers to mirror a folder contents on another (ubuntu) device that frequently creates new files (photos).

 

In this scenario, is folder rescan even necessary on the Raspberry Pi? If not, how do I disable rescanning, simply set folder_rescan_interval to a very large number?