If you are a system administrator, you have probably wondered at least once ohw to configure your Linux server to automatically reboot itself if it crashes, is going through a mass CPU overload, e.g. the server load average "hits the sky".
I just learned from a nice article found here that there is a kernel variable which when enabled takes care to automatically restart a crashed server with the terrible Kernel Panic message we all know.
The variable I'm taking about is kernel.panic for instance kernel.panic = 20 would instruct your GNU Linux kernel to automatically reboot if it experiences a kernel panic system crash within a time limit of 20 seconds.
To start using the auto-reboot linux capabilities on a kernel panic occurance just set the variable to /etc/sysctl.conf
debian-server:~# echo 'kernel.panic = 20' >> /etc/sysctl.conf
Now we will also have to enable the variable to start being use on the system, so execute:
debian-server:~# sysctl -pThere you go automatic system reboots on kernel panics is now on.
Now to further assure yourself the linux server you're responsible of will automatically restart itself on a emergency situation like a system overload I suggest you check Watchdog
You might consider checking out this auto reboot tutorial which explains in simple words how watchdog is installed and configured.
On Debian installing and maintaining watchdog is really simple and comes to installing and enabling the watchdog system service, right afteryou made two changes in it's configuration file /etc/watchdog.conf
To do so execute:
debian-server:~# apt-get install watchdog
debian-server:~# echo "file = /var/log/messages" >> /etc/watchdog.conf
debian-server:~# echo "watchdog-device = /dev/watchdog" >> /etc/watchdog.conf
Well that should be it, you might also need to load some kernel module to monitor your watchdog.
On my system the kernel modules related to watchdog are located in:
/lib/modules/2.6.26-2-amd64/kernel/drivers/watchdog/
If not then you should certainly try the software watchdog linux kernel module called softdog , to do so issue:
debian-server:~# /sbin/modprobe softdog
It's best if you load the module while the softdog daemon is disabled.
If you consider auto loadig the softdog software watchdog kernel driver you should exec:
debian-server:~# echo 'softdog' >> /etc/modules
Finally a start of the watchdog is necessery:
debian-server:~# /etc/init.d/watchdog start
Stopping watchdog keepalive daemon....
Starting watchdog daemon....
That should be all your automatic system reboots should be now on!
Related Posts
Tags: auto reboot, care, configuration file, configure, crash, debian gnu, debian server, emergency, emergency situation, file, gnu linux, How to automatically reboot (restart) Debian GNU Lenny Linux on kernel panic, init, instance, kernel panic, kernel panics, limit, linux capabilities, linux kernel, linux server, log, log messages, modprobe, nbsp, necessery, ohw, quot, server load, sky, software, software watchdog with kernel module softdog, some general CPU overload or system crash, squeeze, Stopping, system administrator, system crash, system overload, time, time limit, tutorial, Watchdog, watchdog system




on
on 
[...] If you are a system administrator, you have probably wondered at least once ohw to configure your Linux server to automatically reboot itself if it crashes, is going through a mass CPU overload, e.g. the server load average “hits the sky”. More here [...]
Great post. Thanks!
[...] How to automatically reboot (restart) Debian GNU Lenny Linux on kernel panic, some general CPU overl… 此条目发表在 忙里偷闲 分类目录,贴了 debian, server 标签。将固定链接加入收藏夹。 ← 我们的新T恤 毕业的煎熬 → [...]
[...] there are plenty of things like Server Auto Reboot on Crash with software Watchdog etc., that we as sysadmins can do to reduce server downtimes and most of the manual human [...]
Thanks for your article. Two questions:
1- If I reboot the server manually (for service), then should I run “sysctl -p” again or only once it is needed?
2- Is it mandatory to use “watchdog” in order for “kernel.panic” to work or it is an independent package?
Hi mandhood,
Here is the answers:
1.If you put the settings inside /etc/sysctl.conf on next reboot you don’t need sysctl -p
2. I think watchdog is not mandatory, however it’s recommended since at many cases the system could freeze without a kernel panic and then you will need reboot.
You mean that if server hangs without kernel panic message, the watchdog can detect that and ‘kernel.panic’ will operate after 20 seconds. Right?
aha
Best!
Georgi
[...] recently blogged on the topic of Debian Linux auto-restart in case of kernel panic , however now i had to conifgure watchdog on some dozen of CentOS Linux [...]
Just a small note it might be better to set the kernel panic to be kernel.panic = 10 or kernel.panic = 1 to restart the hanged server faster
[...] http://www.pc-freak.net/blog/how-to-automatically-reboot-restart-debian-gnu-lenny-linux-on-kernel-pa… for some good background on how this all [...]
[...] is going through a mass CPU overload, e.g. the server load average “hits the sky”. More hereНайдено по поисковым фразам:cpu panicdebian cpu температураdebian [...]
[...] http://www.pc-freak.net/blog/how-to-automatically-reboot-restart-debian-gnu-lenny-linux-on-kernel-pa… for some good background on how this all [...]
[...] the server if oom_killer goes up.watchdog (see this link)is a good [...]