System logging with syslog-ng

Abstract

When operating a production installation the availability and reliability of services, especially those deemed mission-critical, is of primary importance. Most failures, when analysed after the event, could have been prevented had adequate system logging and reporting systems been in place to ensure that the conditions leading up to the failure were noted and remedial action taken. Such automated log analysis tools can only operate effectively if they have a reliable and consistent set of log files to work from.

This document details how to install and configure the syslog-ng system logging daemon to fulfil a variety of requirements ranging from simply collecting the messages generated by a single system and recording them in a text file to receiving the logs from dozens of different machines and storing them in a database for further analysis while sending email notifications of serious events.

Contents

• Preparing the ground
  • Introduction
    • What is logging?
    • Why do I need to log events?
  • Installing the SyslogNG software
    • Patching the package ebuild
    • Installing the package
    • Creating a group and user
    • Configuring the daemon
• Basic logging
  • Local logging
    • Basic options
    • Sources
    • Filters
    • Destinations
    • Mappings
    • Testing
  • Log rotation
    • Why rotate my logs?
    • Rotation methods
    • Rotating logs using syslog-ng macros
    • Compressing and deleting logs
• Intermediate logging
  • Running a log server
    • Why run a log server?
    • TCP or UDP
    • Network logging options
    • Using macros to further organise log files
  • Logging to a remote log server
    • TCP or UDP
    • What to log to the server
    • What to log locally
• Advanced logging
  • Email notification of critical events
    • Notification methods
    • The program driver
    • The pipe driver
  • Logging to a database
    • Why log to a database?
    • Creating the database
    • Database connection