Logging to a remote log server

TCP or UDP

When we described, in the previous chapter, how to configure a syslog-ng log server to receive log messages over the network we had to decide which protocols we were going to listen on, either TCP, UDP or both. Assuming that you have read that section you should already know the advantages and disadvantages of each protocol so there is no point discussing them again here. We shall, therefore, simply provide example destination configurations for both protocols.

TCP

As you can see from the example below creating a destination to send log messages over TCP is trivial. The only required parameter is the name of the log server to which messages should be sent.

destination d_logserver    { tcp("logserver.example.com"); };

As TCP is a connection based protocol a connection to the log server will be established when the syslog-ng daemon starts. Should this connection fail for any reason then the syslog-ng daemon will attempt to reopen the connection after a delay which can be configured using the time_reopen option.

While the connection is down log messages will be stored in a buffer, the size (in messages) of which can be configured using the log_fifo_size option, until the connection is reestablished. If this buffer becomes full then the number of messages which were lost will be recorded and reported in the status line in the syslog-ng log file.

UDP

Creating a destination to send log messages to a log server using the UDP protocol is almost exactly the same as when using the TCP protocol. The only difference is that we use the udp output driver instead of the tcp driver. As before the name of the log server is the only required parameter.

destination d_logserver    { udp("logserver.example.com"); };

Unlike TCP the UDP protocol has no concept of connections. For this reason there is no need for the connection to be reopened if the log server goes down. In such an event messages will continue to be sent just as if the log server was operating normally and will, therefore, be permanently lost with no obvious evidence that this ever occurred.

What to log to the server

When logging to a remote log server some care needs to be taken to ensure that only the log messages which we are interested in will be transmitted over the network. If we just direct all log messages to the server we could very quickly saturate our available network bandwidth should debug level logging be enabled on a busy service.

We can create a simple mapping which will direct all non-trivial log messages, which we have defined as everything which is not of debug or info level, to the server as shown below.

log { source(s_all);    filter(f_not_trivial);    destination(d_logserver); };

This one mapping should ensure that all error messages and warnings are sent to the log server while everything else is ignored. What we do with any log messages which are not sent to the log server will be covered in a later section.

Sending log messages over the network can pose a serious security risk depending on their content. It is highly recommended therefore that all network logging is encapsulated over a secure protocol such as IPsec or an SSH tunnel.

 

In addition to the non-trivial messages which we send to the server using the mapping above there are a number of info and debug level messages which may be of interest in a production environment. The most important of these are probably those logged with the auth and authpriv facility codes. We can add two more mappings, as shown below, to include these messages amongst those which we send to the log server.

log { source(s_all);    filter(f_auth);
                        filter(f_trivial);        destination(d_logserver); };

log { source(s_all);    filter(f_authpriv);
                        filter(f_trivial);        destination(d_logserver); };

As you can see in the above example we have used an additional filter on each mapping to ensure that non-trivial messages will not be sent again as they will have already been sent by our previous mapping. An alternative option would have been to use the fallback flag instead.

 

What to log locally

Network failures! Syslog messages! Debug messages! Fallback!