Internet Protocol Security (IPsec)

The work on the IP security suite protocol (IPsec) is carried out in the ipsec working group (www.ietf.org/html.charters/ipsec-charter.html) in the security area of the IETF. We strongly recommend Kaufman and Newman (1999) if you would like a detailed treatment of the subject and its business applications.

RFC 2401 defines the design goal of the IP security (IPsec) protocol as the means to provision interoperable, high-quality, cryptography-based security for both versions of IP protocol—IPv4 and IPv6. In a nutshell, the IPsec suite provides privacy (through encryption) and authentication services at the IP layer (which, among other things, means that application and transport security as well as data link layer security are outside of the scope of the IPsec suite). The services provided by the suite include access control, connectionless integrity, confidentiality, and protection against replays (that is, reuse of snooped messages).

To provide these services, first, two traffic security protocols are defined:

§  Authentication Header (AH) (RFC 2402). Provides connectionless integrity, data origin authentication, and an (optional) antireplay service.

§  Encapsulating Security Payload (ESP) (RFC 2406). Provides one of the two sets of services:

1.  Confidentiality and limited traffic flow confidentiality.

2.  Integrity, data origin authentication, and an antireplay service.

Each of these protocols can be used in either transport mode (where transport protocols are protected) or tunnel mode (where the IP itself is protected). Tunnel mode is particularly important to VPN services in connection with IPIP (RFC 2003).

Second, algorithms for encryption and authentication as well as automatic key management schemes (such as RFC 2408 and RFC 2409) are defined, each in a separate RFC. The informational RFC 2411 provides guidelines for specifying new encryption and authentication algorithms and explains the interrelationship of all the protocols and algorithms in the IPsec suite.

The interrelationship of the elements of the IPsec architecture is demonstrated in Figure 1. The ESP and AH protocol documents cover the packet format and general issues of these protocols. They also define the default and mandatory values (as, for example, specified in RFC 2407) that are fed into the domain of interpretation (DOI) document, which deals with assigned values. The ESP protocols can use various encryption algorithms (for example, those specified by RFC 2405 and RFC 2451) as well as authentication algorithms (for example, those specified by RFC 2403 and RFC 2404). The authentication algorithms are also used by the AH protocol.

Figure 1: IPsec architecture.

Source: RFC 2411.

Both the encryption and authentication algorithms may also specify values for the DOI (for example, the values that identify these algorithms). Finally, the key management documents specify the IETF standards-track key management schemes, which also define the values for DOI. The IPsec DOI also supports negotiation of IP compression (as specified in RFC 2393); this function may be necessary because the encryption of IP payload prevents compression at lower layers. For example, a message containing a string of 50 octets filled with 0s could have been successfully compressed to something much smaller than 50 octets had it not been for the encryption, which—by design—not only changed 0s to other values but also eliminated the pattern.

Overall, IPsec provides security services at the IP layer by selecting (1) appropriate security protocols, (2) algorithms to be used with these protocols, and (3) appropriate cryptographic keys. The major function of the Internet Key Exchange (IKE) protocol (RFC 2409) is to establish and manage security associations (SAs), a function supported by both AH and ESP.

The concept of SAs is fundamental to IPsec. RFC 2401 defines an SA as a simplex connection that affords security services to the traffic carried by it. These services can be afforded by either AH or ESP, but in the case where both are to be employed, two SAs need to be established. An SA is uniquely identified by the following triplet:

1.  Security parameter index (SPI).

2.  IP destination address. Although this can in principle also be a broadcast or multicast address, only the unicast address is presently supported by the IPsec management mechanism.

3.  Security protocol identifier (that is, AH or ESP).

RFC 2401 defines two types of SAs—transport mode SA and tunnel mode SA—to support the AH and ESP modes of operation, respectively. As Figure 2 demonstrates, the transport mode SA is an association between two hosts. The double arrows are used only to indicate the nodes between which the SAs are established, not to imply by any means that that the SAs are duplex! The tunnel mode SA, however, may be established between:

§  Two security gateways. A security gateway is defined in RFC 2401 as an intermediate system that acts as the communications interface between two networks.

§  A host and a security gateway.

§  Two hosts.

Figure 2: Two types of security associations (SAs).

Any SA that supports tunneling (that is, an SA between a host and a security gateway or two security gateways) must be a tunnel mode SA, but an SA between two hosts may be either a transport mode SA or tunnel mode SA. The modes define the order and selection of appropriate headers. RFC 2401 provides several interesting examples of how SAs can be combined to satisfy complex, nesting security policies, and introduces into the model the notions of the security policy database (which specifies the policies that determine the disposition of IP traffic) and the security association database (which specifies the parameters of each active SA). Finally, RFC 2401 studies four cases of security associations, of which the fourth—depicted in Figure 3—is specifically relevant to the topic of dial-in access to an enterprise network. 

Figure 3: Remote host reaching an enterprise server.

Source: RFC 2401, Case 4.

In Figure 3, a remote host (dialing into an access concentrator) uses the Internet to reach an enterprise’s firewall (a security gateway) and then a particular host (a server). Only tunnel mode is required for an SA between the remote host and the firewall, while an SA between the remote host and the server can be either a transport or tunnel mode SA.

We have mentioned how IPsec standards relate to access and, particularly, VPNs, but so far we have not said anything about IPsec’s relation to IP telephony. The general feeling in the IP telephony community^[2] is that AH and ESP can be used in a standalone IP telephone, but the complexities of key exchange through IKE and the Internet Security Association and Key Management Protocol (ISAKMP) are too much to be handled by a client in a small standalone telephone.

Kaufman and Newman (1999) report that “early generations of IPsec products have proven to be suboptimal for many VoIP installations” and recommend deploying IPsec and voice over Internet Protocol (VoIP) separately and “experiment[ing] with integration only when you have debugged both installations.” In addition to technical reasons, the same monograph brings up an important regulatory consideration: In some circumstances, there are regulations that restrict voice encryption.

Authentication, Authorization, and Accounting (AAA)

Authentication, authorization, and accounting have become so important that the IETF has started a separate working group (www.ietf.org/html.charters/aaa-charter.html ) that is focusing on a general AAA architecture for the Internet in order to “create a set of base protocols applicable to a number of specific AAA applications.” The applications include IP telephony and access server AAA. Note that the IETF does not get involved in the issues of business models or billing; the first two As were standardized as part of Remote Authentication Dial-In User Service (RADIUS), defined in RFC 2138, while the accounting part is published in the informational RFC 2139. RADIUS has been implemented and deployed in remote access servers ; the aaa working group was created as a follow-up to RADIUS.

Authentication of an entity can be done via a Password Authentication Protocol (PAP), which sends the request for password and then verifies the response, or a Challenge Handshake Authentication Protocol (CHAP), which “challenges” the entity to be authenticated by sending it a random string S.

The entity uses a function f (identified by a secret key, which may otherwise be known only to the authenticator) to compute f(S). The computed value is sent to the authenticator, who computes the same function on the same string, checks the response, and confirms or fails the entity depending on whether the computed and received values are equal. In a communication session, this process can repeat at random intervals.

PPP CHAP is standardized in RFC 1994, which, among other things, provides a thorough review of its advantages and disadvantages (there are some) as well as some practical considerations. RFC 1994 specifies four messages: Challenge, Response, Success, and Failure. Challenge is sent during the authentication phase (but it may be sent at other times, too); it is sent again and again until a satisfactory Response message is received (or an optional counter exceeds a specified value). The Failure message is sent if the Response value is unsatisfactory; otherwise, naturally, the Success message is sent.

Although authentication is performed in only one direction, the processes on the both ends of the PPP link can use it to authenticate each other. RFC 1994 effectively requires that Message Digest Algorithm Five (MD5) (RFC 1321) be supported for hashing with CHAP. In practice, either specialized hardware (typically, a card) or software is employed to compute the challenge. Although we don’t cover cryptography in detail, it is worth at least explaining why the words message digest are used in the name of the algorithm. This happens because the algorithm, which takes a message M of any length as an input, produces as the output a string MD5(M), whose length is fixed. Computing this string is much faster than encrypting the whole message. For more information on cryptography, see Tanenbaum (1996).

RFC 2138 defines RADIUS as a client-server protocol. One client of the RADIUS server is the Network Access Server (NAS), which can perform the LAC or LNS function (or both). In turn, a RADIUS server may also act as a proxy client to other RADIUS or (non-RADIUS) authenticating servers. When a user requests a connection, the NAS makes a corresponding request to an appropriate RADIUS server, which then attempts to authenticate the users. If the authentication is successful, the RADIUS server sends to the NAS the configuration information pertinent to the user in order to establish the requested connection and otherwise provide all services that the user is entitled to. Specifically, RADIUS server queries the user database, whose entries include lists of requirements (such as password verification) to be met in order to establish the connection. The user database entries may contain specific resources (for example, servers) that a particular user may access as well as specific port numbers.

All transactions between the client and server are authenticated through the use of the shared secret, which is never sent over the network. (The users’ passwords, however, are sent to RADIUS servers over the network, but they are always encrypted before they are sent.) The shared secret is put through the MD5 algorithm to create a 16-octet-long digest value used to authenticate messages between the RADIUS client and RADIUS server. Before we continue our discussion of the RADIUS messages, note that ultimately it is the NAS that is responsible for authenticating the end user (with or without RADIUS). Thus the NAS is the first entity to receive the user’s password. (This is done by either prompting the user or employing the information that arrived in the PPP authentication packets.)

Once the NAS has the password, it may submit it to a RADIUS server in the Access-Request message, whose attributes include the user’s ID and password (hidden using the MD5 algorithm). The RADIUS server considers such a request valid only if it shares a secret with the client. All invalid Access-Request messages are discarded; every valid Access-Request message is responded to (after a lookup in the user database) with one of the following three messages:

§  Access-Reject. This message is issued if the requirements pertinent to the user requesting access are not met.

§  Access-Accept. This message is issued if such requirements are met.

§  Access-Challenge. This message is issued if the user is to be authenticated via the challenge mechanism.

The latter message may include the text (typically obtained from an external server) to be displayed to the user (for example, Challenge 90778976. Please respond at the prompt =>). The NAS issues the request to the user, collects the response, encrypts it, and sends it back to the RADIUS server in the password field of yet another Access-Request message. The RADIUS server may then issue another challenge or reply with either Access-Reject or Access-Accept, depending on whether the user’s response to the challenge matches the expected response.

For the cases where PAP and CHAP are employed by the NAS, RFC 2138 specifies interworking with these protocols, which in most cases amounts to one-to-one mapping of respective attributes. In some cases, however, the RADIUS server may be unable to perform the requested CHAP-compliant authentication. In the example given in RFC 2138, the user password may be unknown to the RADIUS server in cleartext. Since CHAP needs the cleartext password value in order to encrypt the CHAP challenge, RADIUS cannot perform the authentication. RFC 2138 leaves no ambiguity, however, by requesting that the RADIUS server send Access-Reject when it cannot go through with authentication.

The remaining RADIUS messages are:

§  Accounting Request and Accounting Response. Described in the informational RADIUS RFC (2139), although their codes are assigned by RFC 2138.

§  Status-Server and Status-Client. Designated experimental by RFC 2138.

All the RADIUS messages are assigned codes specified in RFC 2138. In addition, one RADIUS message code is reserved for the future.

RFC 2138 has deliberately chosen UDP as the transport layer for RADIUS messages. Four reasons for doing so include:

1.  Presence of alternative servers.

2.  Timing requirements.

3.  Stateless nature of the protocol.

4.  Simplification of server implementation.