Sync with Raspberry Pi


HeinerDD

Recommended Posts

Hello,

I try to use Sync as a backup on Raspberry Pi 2. I use the newest version of Sync on Raspberry Pi and the Windows client.

I have to sync about 180.000 files (730 GByte). After a while of indexing some files are synced, but then the synicing stops without any error message until I reboot the Pi.

Can the Pi 2 handle so many files or do I have to buy a "bigger" server?

Thanks for answers.

Heiner

Link to post
Share on other sites
  • Replies 50
  • Created
  • Last Reply

Memory-wise, it should be fine. Sync uses about 0.8-1.5KB per file, so taking the upper bound thats ~263 MB RAM. Since the Pi 2 has 1GB of memory, that shouldn't cause many problems (unless you are running other large applications). Did you check:

- If the btsync process is still running?

- If it is: how many resources are used (CPU time, memory), e.g. using 'top'.

- Sync's log file?

Link to post
Share on other sites

Thank you for your prompt answer.

I figured out, that the ntfs-3g needed a lot of prozessor power (more than 30%). I formated the HDD with ext4. Now the tranfer speed is much better, but the the btsync process crashed again after half a day.

I cant find the btsync process in "ps aux"-list. The last entries (and many more the same) in sync.log are:

[20160206 21:42:40.016] [OnNotifyFileChange] "/media/hd/h/Eigene Bilder/Digi-Cam/2012_08_15/C56A0551.CR2.bts"
[20160206 21:42:41.038] TF[3AE4] [0x61ef7ab8][/media/hd/h/Eigene Bilder/Digi-Cam/2012_08_15/C56A0541.JPG]: closed all handles, fs_refs = 1
[20160206 21:42:41.089] SyncFolderNotify: SyncFolderNotify: "C56A0551.CR2.bts", event = "IN_MODIFY"
[20160206 21:42:41.133] TF[3AE4] [0x6a5f41e0][/media/hd/h/Eigene Bilder/Digi-Cam/2012_08_15/C56A0542.CR2]: closed all handles, fs_refs = 1
[20160206 21:42:41.141] [OnNotifyFileChange] "/media/hd/h/Eigene Bilder/Digi-Cam/2012_08_15/C56A0551.CR2.bts"
[20160206 21:42:41.163] TF[3AE4] [0x2fe2a700][/media/hd/h/Eigene Bilder/Digi-Cam/2012_08_15/C56A0551.CR2]: piece 42 complete
[20160206 21:42:41.163] SyncFolderNotify: SyncFolderNotify: "C56A0551.CR2.bts", event = "IN_MODIFY"
[20160206 21:42:41.163] [OnNotifyFileChange] "/media/hd/h/Eigene Bilder/Digi-Cam/2012_08_15/C56A0551.CR2.bts"
[20160206 21:42:41.164] SyncFolderNotify: SyncFolderNotify: "C56A0551.CR2.bts", event = "IN_MODIFY"
[20160206 21:42:41.164] [OnNotifyFileChange] "/media/hd/h/Eigene Bilder/Digi-Cam/2012_08_15/C56A0551.CR2.bts"
[20160206 21:42:41.164] SyncFolderNotify: SyncFolderNotify: "C56A0551.CR2.bts", event = "IN_MODIFY"
[20160206 21:42:41.165] [OnNotifyFileChange] "/media/hd/h/Eigene Bilder/Digi-Cam/2012_08_15/C56A0551.CR2.bts"
[20160206 21:42:41.165] SyncFolderNotify: SyncFolderNotify: "C56A0551.CR2.bts", event = "IN_MODIFY"
[20160206 21:42:41.166] [OnNotifyFileChange] "/media/hd/h/Eigene Bilder/Digi-Cam/2012_08_15/C56A0551.CR2.bts"
[20160206 21:42:41.166] SyncFolderNotify: SyncFolderNotify: "C56A0551.CR2.bts", event = "IN_MODIFY"
[20160206 21:42:41.166] [OnNotifyFileChange] "/media/hd/h/Eigene Bilder/Digi-Cam/2012_08_15/C56A0551.CR2.bts"
[20160206 21:42:41.167] SyncFolderNotify: SyncFolderNotify: "C56A0551.CR2.bts", event = "IN_MODIFY"
[20160206 21:42:41.167] [OnNotifyFileChange] "/media/hd/h/Eigene Bilder/Digi-Cam/2012_08_15/C56A0551.CR2.bts"
[20160206 21:42:41.168] SyncFolderNotify: SyncFolderNotify: "C56A0551.CR2.bts", event = "IN_MODIFY"
[20160206 21:42:41.168] [OnNotifyFileChange] "/media/hd/h/Eigene Bilder/Digi-Cam/2012_08_15/C56A0551.CR2.bts"
[20160206 21:42:41.169] SyncFolderNotify: SyncFolderNotify: "C56A0551.CR2.bts", event = "IN_MODIFY"
[20160206 21:42:41.169] [OnNotifyFileChange] "/media/hd/h/Eigene Bilder/Digi-Cam/2012_08_15/C56A0551.CR2.bts"
[20160206 21:42:41.169] SyncFolderNotify: SyncFolderNotify: "C56A0551.CR2.bts", event = "IN_MODIFY"
[20160206 21:42:41.169] [OnNotifyFileChange] "/media/hd/h/Eigene Bilder/Digi-Cam/2012_08_15/C56A0551.CR2.bts"
[20160206 21:42:41.170] SyncFolderNotify: SyncFolderNotify: "C56A0551.CR2.bts", event = "IN_MODIFY"
[20160206 21:42:41.170] [OnNotifyFileChange] "/media/hd/h/Eigene Bilder/Digi-Cam/2012_08_15/C56A0551.CR2.bts"
[20160206 21:42:41.172] SyncFolderNotify: SyncFolderNotify: "C56A0551.CR2.bts", event = "IN_MODIFY"
[20160206 21:42:41.172] [OnNotifyFileChange] "/media/hd/h/Eigene Bilder/Digi-Cam/2012_08_15/C56A0551.CR2.bts"
[20160206 21:42:41.246] SyncFolderNotify: SyncFolderNotify: "C56A0551.CR2.bts", event = "IN_MODIFY"
[20160206 21:42:41.246] [OnNotifyFileChange] "/media/hd/h/Eigene Bilder/Digi-Cam/2012_08_15/C56A0551.CR2.bts"

Thanks for your help!

Heiner

 

Link to post
Share on other sites

Ok so now we know three things:

1.) After the crash of BitTorrent Sync your Raspberry Pi was not able to restart the process so that is why the first screenshot gave you "-p requires an argument". That argument would've been the PID of the BitTorrent Sync process running but it didn't run so there was no PID.

2.) You have a working script that starts BitTorrent Sync after a reboot.

3.) The screenshot shows that your BitTorrent Sync installation is not really under heavy load.. The system is up for 1 min and normally things would index etc and you would see a high CPU and RAM usage but yours looks idling just ok.

Did you change any of the advance settings on your Raspberry Pi ?

Judging from the file extension .bts you have Selective Sync enabled?

Link to post
Share on other sites

Ok so now we have high CPU and decent RAM usage.. what is the WebGUI showing? Is it transferring data or Indexing?

Have you tried removing and re-adding the share you want to sync the 700GB+ files over?

If everything fails, maybe it is now time to send in the log files to Sync Support and have them take a look..

Link to post
Share on other sites

Crashes sporadically happened with Sync on my Pi around 2.0.x. Since then I switched to systemd, which can automatically restart processes on unsuccessful exits. This is not a solution, but could be a workaround if you want to ensure that Sync is always running.

Your log does not show anything interesting near the end. I think support would be interested in debugging this problem further. You can contact them from within Sync, attaching the logs as well.

Link to post
Share on other sites
21 hours ago, iswrong said:

Crashes sporadically happened with Sync on my Pi around 2.0.x. Since then I switched to systemd, which can automatically restart processes on unsuccessful exits. This is not a solution, but could be a workaround if you want to ensure that Sync is always running.

Your log does not show anything interesting near the end. I think support would be interested in debugging this problem further. You can contact them from within Sync, attaching the logs as well.

Isn`t the laste Raspbian based on Jessie and comes with systemd?

Link to post
Share on other sites
21 hours ago, Moe said:

Ok so now we have high CPU and decent RAM usage.. what is the WebGUI showing? Is it transferring data or Indexing?

Have you tried removing and re-adding the share you want to sync the 700GB+ files over?

If everything fails, maybe it is now time to send in the log files to Sync Support and have them take a look..

Where can I send the log file?

Link to post
Share on other sites
7 hours ago, HeinerDD said:

If the Raspi is unstable, shouldnt then other processe also be affected?

Power supply problems can lead to memory corruption on the Pi and probably Sync is one of the processes using most of its memory.

https://www.raspberrypi.org/forums/viewtopic.php?p=589877#p589877

http://elinux.org/R-Pi_Troubleshooting#Troubleshooting_power_problems

Not saying that this is the problem, but it can be a problem. But it's relatively easy to check whether the power source is good.

Link to post
Share on other sites

@HeinerDD Do I understand correctly that if you start sync after setting ulimit it

  1. Is not visible by other peers
  2. Consumes very small amount of memory and CPU time and does not introduce same behavior as before?

If yes - it looks like you start it differently from how you start it before and Sync simply has taken a different storage path. How do you start it usually?

Link to post
Share on other sites

Hi again!

After about 20 reboots, the syncing seems to be complete.  The btsync-process is still randomly killed, but I cant get a coredump to open a Support case.

I think its not the Power supply. I changed it. The Raspi gots its own. The HD is connected to a powerde USB-HUB.

I don`t know what to do.

Link to post
Share on other sites

@HeinerDD

It's weird. Could you please check the content of a file "/proc/sys/kernel/core_pattern" It should contain a path and a mask for core dump files created. Any core dumps where it points to?

Also, what about running Sync with ulimit set to unlimited? Last time you responded it behaves like it started a fresh instance of Sync without any old folders. Is it still true? Did you run it with completely same command line as you did before?

Link to post
Share on other sites

I have changed the /proc/sys/kernel/core_pattern to

/tmp/cores/core.%e.%p.%h.%t

In /tmp/cores/ are no files after the crash.

In /tmp/cores/ is another directory called /btsync_dumps, but its also empty.

The problem with low cpu and RAM usage was my fault, I did not start btsync with the init.d-script I usually use.

Link to post
Share on other sites

Archived

This topic is now archived and is closed to further replies.