IPsec is a technology widely known. Created for IPv6 and backported to IPv4, it adds a security layer into the IP stack. Prior to IPsec we needed to encrypt data before sending it down the stack and then decrypt it on the other side once it came back out using technologies like SSH or SSL/TLS. IPsec simplifies this by transparently handling encrypt/decrypt as well as header authentication.

But for all its intended simplicity and transparency it is to the administrator anything but. If you google around you find piles of email on various lists of those who tried to get IPsec working and gave up in frustration and instead used OpenVPN, on both Linux and Solaris.

Part of the reason IPsec is so complex to manage is the number of technologies that all must work together properly in order to get something functional. This includes acronyms like AH, ESP, IKE, DH, RSA, SA, SAD, etc. In many respects its similar to the frustration encountered the first time you approach LDAP and are overwhelmed with OU=, CN=, DN=, etc. Just like LDAP, with a little practice you get it sorted out and start making ground. We’ll start high level and zoom in… please please, read this before just copying examples! (I know you won’t, but it’ll save you time.)

IPsec is fundamentally similar to a firewall, in that you specify IPsec Policies that determines how datagrams are handled. If a policy matches a datagram its handled… if no policy matches it acts like normal.

Thats where the firewall analogy stops, like a firewall the policy says what to do and when to do it…. but we need to know something else for encrypting or decrypting data, we also need to know how to do it. The information that explains the how is known as a Security Association (SA). This is maintained in a special database named, unremarkably, the Security Association Database (SAD or SADB, depending on who you ask, same thing though).

It’s the SA that actually contains the keys used for encrypting, decrypting and authenticating IPsec datagrams. So a packets goes through the stack, a policy says “If the datagram source is 1.2.3.4 and the destination is 5.6.7.8 use IPsec.” Now that the stack knows to use IPsec to encrypt, it goes looking in the SADB for an SA that contains the keys. There is a lot of duplication here because they are independent things, so you specify the authentication algorithm, encryption algorithm, source and destination addresses in both the IPsec policy and the SA. Its strange at first. 🙂

Now, just as IPsec is supposed to make life simple and transparent, so is IKE (Internet Key Exchange). An IKE daemon runs on both sides of a connection and negotiates SA’s for you. This provides a variety of benefits, but simplicity isn’t exactly one of them because you still need to configure IKE rules which are similar to SA’s. In other words, you can’t just create an IPsec policy, and then enable IKE and be done.

Before we get into the examples, lets looks at the various files and commands involved:

Files:

    * IPsec:
          o /etc/inet/ipsecinit.conf: IPsec Policy Definitions
          o /etc/inet/secret/ipseckeys: IPsec SA Definitions 
    * IKE:
          o /etc/inet/ike/config: IKE Global Configuation and Rules
          o /etc/inet/secret/ike.preshared: File containing Preshared Key Definitions
          o /etc/inet/secret/ike.privatekeys/: Directory containing IKE Private Keys
          o /etc/inet/ike/publickeys/: Directory containing IKE Public Keys
          o /etc/inet/ike/crls/: Directory containing IKE Certificates 

Commands:

    * IPsec:
          o ipsecconf: Load or Display IPsec Policy Configuration
          o ipsekkey: Manually manipulate IPsec Security Association Database (SADB)
          o ipsecalgs: Display available IPsec ESP/AH Algorithms 
    * IKE:
          o in.iked: IKE Daemon
          o ikeadm: Manipulate IKE parameters and state (flush, add, get, set, ...)
          o ikecert: Manipulate IKE's on-filesystem public-key certificate databases 

The simplest configuration would be to “pre-share” keys, meaning create a key and manually put it on both systems. In my next blog entry we’ll step through actually creating an IPsec policy, creating keys, creating Security Associations, and testing an IPsec connection.