Email notification of critical events

Notification methods

Although having ones log messages logged to files is a good start there are some events about which you may want to be immediately notified. Such events may include running out of disk space on a partition, a user logging in to a server which should not accept shell logins, a user using the su command to become root, etc.

As we saw in previous sections the syslog-ng daemon provides a number of useful output drivers which can be used when logging events. Some of these output methods can be used, with varying degrees of stability and complexity, to send log messages to an administrator by email.

In the following sections we shall examine two possible alternative methods of sending email notification of events to specified users.

Care should be taken when configuring the syslog-ng daemon to send email alerts to ensure that any log events which will be generated in response to a failure of the email sending mechanism will not in turn generate email notifications of their own.

 

The program driver

This is probably the easiest to implement method of sending email notifications. It makes use of a simple bash script, to send the emails, and the program output driver, to start the bash script and send it the log messages.

The first step is to create a very simple bash script that will read lines of text from stdin and send them, by email, to an administrator. Start by editing a new file using nano as shown below.

Enter the following shell script, changing the email address and subject as appropriate.

#!/bin/bash

subject="Replace with a sensible subject"
email="someone@example.com"

while read event;
do
  echo ${event} | mail -s "${subject}" ${email}
done

And make the script executable by entering the following command.

The next step is to create an output destination which uses the program driver to execute our script by adding a line like that shown below to the configuration file.

destination d_email_notification { program("/usr/local/bin/log-emailer.sh"); };

Despite its simplicity this method has several weaknesses. The most glaring in my mind is that it requires the syslog-ng daemon to be able to fork shell scripts, something which I would deem an unnecessary security risk, especially if the syslog-ng daemon is being run as the root user.

The pipe driver

The next easiest to implement method would be to use the pipe output driver along with the same shell script we developed earlier. This method avoids the most serious problem with the program method above as it reads log messages from a named pipe rather than being executed directly.

Assuming that we have already created the above script and made it executable then all that we need to do to implement this method is create a pipe and start our script reading from it, as shown below.

We can then direct log messages to this pipe by adding a line such as that shown below to our configuration file.

destination d_email_notification { pipe("/var/log/email-pipe"); };

This method neatly solves the problem of having to allow the syslog-ng process to execute shell scripts, as well as having the script need to run as the same user as the syslog-ng daemon. Should the script stop running for any reason then syslog-ng will attempt to reconnect to the pipe every 60 seconds, although this behaviour can be modified using the time_reopen option.