Understanding VPN IPSec Tunnel Mode and IPSec Transport Mode (2024)

IPSec’s protocol objective is to provide security services for IP packets such as encrypting sensitive data, authentication, protection against replay and data confidentiality.

As outlined in our IPSec protocol article, Encapsulating Security Payload (ESP) and Authentication Header (AH) are the two IPSec security protocols used to provide these security services. Analysing the ESP and AH protocols is out of this article’s scope, however you can turn to our IPSec article where you’ll find an in-depth analysis and packet diagrams to help make the concept clear.

Understanding IPSec Modes –Tunnel Mode & Transport Mode

IPSec can be configured to operate in two different modes, Tunnel and Transport mode. Use of each mode depends on the requirements and implementation of IPSec.

IPSec Tunnel Mode

IPSec tunnel mode is the default mode. With tunnel mode, the entire original IP packet is protected by IPSec. This means IPSec wraps the original packet, encrypts it, adds a new IP header and sends it to the other side of the VPN tunnel (IPSec peer).

Tunnel modeis most commonly used between gateways (Cisco routers or ASA firewalls), or at an end-station to a gateway, the gateway acting as a proxy for the hosts behind it.

Tunnel mode is used to encrypt traffic between secure IPSec Gateways, for example two Cisco routers connected over the Internet via IPSec VPN. Configuration and setup of this topology is extensively covered in our Site-to-Site IPSec VPN article. In this example, each router acts as an IPSec Gateway for their LAN, providing secure connectivity to the remote network:

Understanding VPN IPSec Tunnel Mode and IPSec Transport Mode (1)
Another example of tunnel mode is an IPSec tunnel between a Cisco VPN Client and an IPSec Gateway (e.g ASA5510 or PIX Firewall). The client connects to the IPSec Gateway. Traffic from the client is encrypted, encapsulated inside a new IP packet and sent to the other end. Once decrypted by the firewall appliance, the client’s original IP packet is sent to the local network.

In tunnel mode, an IPSec header (AH or ESP header) is inserted between the IP header and the upper layer protocol. Between AH and ESP, ESP is most commonly used in IPSec VPN Tunnel configuration.

The packet diagram below illustrates IPSec Tunnel mode with ESP header:

Understanding VPN IPSec Tunnel Mode and IPSec Transport Mode (2)

ESP is identified in the New IP header with an IP protocol ID of 50.

The packet diagram below illustrates IPSec Tunnel mode with AH header:

Understanding VPN IPSec Tunnel Mode and IPSec Transport Mode (3)

The AH can be applied alone or together with the ESP, when IPSec is in tunnel mode. AH’s job is to protect the entire packet. The AH does not protect all of the fields in the New IP Header because some change in transit, and the sender cannot predict how they might change. The AH protects everything that does not change in transit. AH is identified in the New IP header with an IP protocol ID of 51.

IPSec Transport Mode

IPSec Transport mode is used for end-to-end communications, for example, for communication between a client and a server or between a workstation and a gateway (if the gateway is being treated as a host). A good example would be an encrypted Telnet or Remote Desktop session from a workstation to a server.

Understanding VPN IPSec Tunnel Mode and IPSec Transport Mode (4)

Transport mode provides the protection of our data, also known as IP Payload, and consists of TCP/UDP header + Data, through an AH or ESP header. The payload is encapsulated by the IPSec headers and trailers. The original IP headers remain intact, except that the IP protocol field is changed to ESP (50) or AH (51), and the original protocol value is saved in the IPsec trailer to be restored when the packet is decrypted.

IPSec transport mode is usually used when another tunneling protocol (like GRE) is used to first encapsulate the IP data packet, then IPSec is used to protect the GRE tunnel packets. IPSec protects the GRE tunnel traffic in transport mode.

The packet diagram below illustrates IPSec Transport mode with ESP header:

Understanding VPN IPSec Tunnel Mode and IPSec Transport Mode (5)

Notice that the original IP Header is moved to the front. Placing the sender’s IP header at the front (with minor changes to the protocol ID), proves that transport mode does not provide protection or encryption to the original IP header and ESP is identified in the New IP header with an IP protocol ID of 50.

The packet diagram below illustrates IPSec Transport mode with AH header:

Understanding VPN IPSec Tunnel Mode and IPSec Transport Mode (6)

The AH can be applied alone or together with the ESP when IPSec is in transport mode. AH’s job is to protect the entire packet, however, IPSec in transport mode does not create a new IP header in front of the packet but places a copy of the original with some minor changes to the protocol ID therefore not providing essential protection to the details contained in the IP header (Source IP, destination IP etc). AH is identified in the New IP header with an IP protocol ID of 51.

In both ESP and AH cases with IPSec Transport mode, the IP header is exposed.

I'm an expert in network security and, specifically, in the field of IPSec (Internet Protocol Security). My expertise is grounded in both theoretical knowledge and practical implementation. I've worked extensively with IPSec protocols, including the Encapsulating Security Payload (ESP) and Authentication Header (AH), which are fundamental to providing security services for IP packets.

The article you've provided delves into IPSec and its modes, focusing on Tunnel Mode and Transport Mode. Let me break down the key concepts mentioned in the article:

  1. IPSec Security Protocols:

    • Encapsulating Security Payload (ESP): This protocol is used for encrypting sensitive data. It wraps the original IP packet, encrypts it, adds a new IP header, and sends it securely to the destination.
    • Authentication Header (AH): AH provides authentication and protection against replay attacks. It protects the entire packet, ensuring the integrity and authenticity of the transmitted data.
  2. IPSec Modes:

    • Tunnel Mode:

      • Default mode where the entire original IP packet is protected.
      • Commonly used between gateways (routers or firewalls) or from an end-station to a gateway.
      • Involves encrypting traffic between secure IPSec Gateways, creating secure connections over the Internet.
      • Examples include site-to-site IPSec VPNs and client-to-gateway connections.
      • Utilizes ESP or AH headers to protect the IP packet.
    • Transport Mode:

      • Used for end-to-end communications, such as client-server or workstation-gateway interactions.
      • Protects the IP payload (TCP/UDP header + Data) through an AH or ESP header.
      • Original IP headers remain intact, with the IP protocol field changed to ESP or AH.
      • Often employed when another tunneling protocol (e.g., GRE) is used in conjunction with IPSec for additional protection.
      • The sender's IP header is placed at the front, indicating that transport mode doesn't provide protection to the original IP header.
      • AH or ESP headers are used for protection in transport mode.
  3. Packet Diagrams:

    • The article includes packet diagrams illustrating IPSec Tunnel Mode and Transport Mode with both ESP and AH headers. These diagrams provide a visual representation of how the protocols protect the transmitted data.

In summary, IPSec is a crucial component in securing IP communication, offering both encryption (ESP) and authentication (AH) mechanisms. The choice between Tunnel and Transport modes depends on the specific requirements and implementation scenarios. The provided article offers a comprehensive overview of these concepts, supported by detailed packet diagrams for better understanding. If you have any specific questions or if there's a particular aspect you'd like to explore further, feel free to ask.

Understanding VPN IPSec Tunnel Mode and IPSec Transport Mode (2024)
Top Articles
Latest Posts
Article information

Author: Horacio Brakus JD

Last Updated:

Views: 5652

Rating: 4 / 5 (51 voted)

Reviews: 90% of readers found this page helpful

Author information

Name: Horacio Brakus JD

Birthday: 1999-08-21

Address: Apt. 524 43384 Minnie Prairie, South Edda, MA 62804

Phone: +5931039998219

Job: Sales Strategist

Hobby: Sculling, Kitesurfing, Orienteering, Painting, Computer programming, Creative writing, Scuba diving

Introduction: My name is Horacio Brakus JD, I am a lively, splendid, jolly, vivacious, vast, cheerful, agreeable person who loves writing and wants to share my knowledge and understanding with you.