Configuring the Racoon IKE/ISAKMP daemon

As we mentioned in the previous section (Managing the Security Association Database using setkey) managing Security Associations (SAs) manually using setkey soon becomes cumbersome when more than a few machines on a network are to be secured. It also limits the range of possible security options as setkey can only use static keys for the encryption and authentication algorithms.

One of the more elegant solutions to these problems, and others, is to make use of the Internet Key Exchange (IKE) protocol (described in RFC-2409◳). The IKE protocol makes use of the framework for authentication and key exchange described by the Internet Security Association and Key Management Protocol (ISAKMP), as well as the OAKLEY Key Determination Protocol (see RFC-2408◳ and RFC-2412◳ for more details) and the SKEME key exchange techniques, to provide a complete solution offering authenticated keying material for use with ISAKMP, and for other security associations such as AH and ESP for the IETF IPsec DOI (described in RFC-2407◳).

In this section we shall describe how the racoon daemon may be used to provide Internet Key Exchange (IKE) services including negotiating security associations for itself (ISAKMP SA, or phase 1 SA) and for kernel IPsec (IPsec SA, or phase 2 SA).

IKE/ISAKMP basics

Before we describe how to configure the racoon daemon we shall briefly describe the basic operation of the Internet Key Exchange (IKE) protocol.

The most important thing to know is that the key exchange takes place in two distinct phases.

• Phase 1 is where the two ISAKMP peers establish a secure, authenticated channel with which to communicate. This is called the ISAKMP Security Association (SA).
• Phase 2 is where Security Associations (SAs) are negotiated on behalf of services such as IPsec or any other service which needs key material and/or parameter negotiation.

There are three key exchange modes which may be used to accomplish a Phase 1 Key Exchange each with particular strengths and weaknesses.

• Main Mode may be used to accomplish a Phase 1 Key Exchange while providing identity protection for both peers. A major limitation of Main Mode is that the network addresses of both peers must be static and known in advance.
• Aggressive Mode may be used to accomplish a Phase 1 Key Exchange where identity protection is not required. As identity protection is not required fewer round trips are needed to complete the key exchange and thus Aggressive Mode is usually slightly faster than Main Mode. Another advantage of Aggressive Mode is that it may be used when one of the clients has a dynamic network address.
• Base Mode is a new Phase 1 Key Exchange mode which allows support for all authentication methods for both static and dynamic network addresses. It also provides some more advanced protection against Denial of Service (DoS) attacks and enhanced negotiation capabilities when compared with Aggressive Mode. For more information see IKE Base Mode◳.

Once the Phase 1 Key Exchange has been completed the two IKE peers have negotiated an authenticated and encrypted communications channel, but only between each other and only for the purposes of further key exchange and parameter negotiation. This channel may now be used to negotiate parameters and keying material which may be used with IPsec Security Associations. This phase of operation is usually referred to as Phase 2 and has a single mode of operation.

• Quick Mode accomplishes Phase 2 key exchanges. As Quick Mode requires a secure channel of communication it is not suitable for performing a Phase 1 key exchange however it is considerably faster than any of the three Phase 1 key exchange modes and offers much greater potential for algorithm and parameter negotiation.

Phase 1 parameters

We mentioned in the previous section that before an IKE peer may be used to negotiate parameters, algorithms and keying material for IPsec they must first establish an authenticated and encrypted channel of communication with another appropriate peer. This is usually referred to as Phase 1 Key Exchange.

The racoon configuration file (usually located at /etc/racoon/racoon.conf) uses the following syntax for specifying a remote Phase 1 peer.

remote name [inherit parent_name] { statements }

As you can see, the first parameter to the remote directive allows a name to be specified for the remote peer. Alternatively, the special name anonymous may be used which will match any peer. Optionally, a named configuration may inherit all the values from a previously declared configuration by specifying the name of the parent configuration after the keyword inherit.

The table below describes some of the more basic configuration statements which may be used inside the remote declaration.

Once the basic parameters for the Phase 1 Key Exchange have been specified one or more proposal sections are required. Each proposal section details the encryption, authentication and compression algorithms which may be used. The proposal syntax is briefly described below.

proposal { sub-statements }

The table below describes some of the more basic configuration statements which may be used inside the proposal declaration.

Phase 2 parameters

Once the Phase 1 Key Exchange has been completed IPsec Security Associations (SAs) may be established. The parameters for the Phase 2 Key Exchange are configured using the sainfo directive.

sainfo (local_id | anonymous) (remote_id | clientaddr | anonymous) [from id_type [string]] [group string] { statements }

As you can see, the syntax of the sainfo directive is fairly complex. In its most simple form the single keyword anonymous will cause the subsequent declaration to be valid for any source and destination address.

Additional configuration options

In addition to the options discussed above there are three more configuration directives which deserve a mention at this stage.

log level;

The first of these controls the level of messages which will be sent to the system log. During configuration and testing a level of info or debug may be required to produce output which may be useful when debugging connection issues. During production use a value of notice or warning is more appropriate to avoid excessively large log files.

path pre_shared_key file;

The next significant configuration directive must be used to configure a path to the Pre Shared Key file which will be used when PSK authentication is selected.

complex_bundle (on | off);

Finally the complex_bundle directive controls how the racoon daemon will interpret proposals when using an SA bundle. When this directive is set to off an ESP tunnel within an outer AH transport will be proposed as AH tunnel and ESP tunnel instead of the more accurate AH transport and ESP tunnel which would be proposed with this directive set to on. For interoperability with older IKE implementations this directive may need to be set to off, which is the default.

Example racoon configuration

Below is a basic example configuration for the racoon daemon.

The first three configuration directives set the log level, the path to the PSK file and the presentation of an SA bundle. As you can see, we have set the log level to info so that more detailed output will be produced as well as specifying that an SA bundle should be expressed accurately.

The next configuration block declares an anonymous remote block which will match any incoming connections. Inside that block we offer a single proposal.

Finally we declare an anonymoussainfo block which will be used to negotiate IPsec Security Associations (IPsec SAs).

log info;

path pre_shared_key "/etc/racoon/psk.txt";

complex_bundle on;

remote anonymous
{
    exchange_mode main;
    my_identifier fqdn "vpn.hacking.co.uk";
    passive off;
    nat_traversal on;
    dpd_delay 30;

    proposal
    {
        encryption_algorithm aes;
        hash_algorithm sha1;
        authentication_method pre_shared_key;
        dh_group modp2048;
        lifetime time 24 hour;
    }
}

sainfo anonymous
{
    encryption_algorithm aes, 3des;
    authentication_algorithm hmac_sha512, hmac_sha384, hmac_sha256, hmac_sha1;
    compression_algorithm deflate;
    pfs_group modp1024;
    lifetime time 12 hour;
}

The above configuration also requires a Pre Shared Key file, an example of which is provided below.

vpn.hacking.co.uk   0x5e0d1ba56cb3249e8a9d1ca243f01da16afc8d24c12f7c45

The PSK file will need to be secured so that it is only readable by the root user before racoon will consider it trustworthy. This can be easily accomplished as shown below.

With the relevant configuration files in place the racoon daemon may be started and added to the default run-level.

In the following section, we shall introduce some techniques for testing and diagnosing common configuration problems with the racoon daemon and IPsec.