What is ESP?

Understanding ESP

ESP (Encapsulating Security Payload) is a core protocol of the IPsec suite that provides confidentiality, data integrity, and authentication for IP packets. Identified as IP protocol number 50 and defined in RFC 4303, ESP encrypts the packet payload so it cannot be read in transit and adds integrity protection so tampering is detectable, making it the primary mechanism for securing VPN traffic.

ESP inserts a header and trailer around the protected data and supports two modes. In transport mode, ESP encrypts and authenticates only the original IP payload, leaving the original IP header intact — used for host-to-host security. In tunnel mode, ESP encrypts and authenticates the entire original IP packet and wraps it in a new IP header — the basis of site-to-site and remote-access VPNs, since it also conceals the internal addressing. ESP provides integrity through an authentication trailer (ICV) and relies on Security Associations and keys negotiated by IKE.

ESP matters because it delivers the encryption that AH (Authentication Header), IPsec's other protocol, lacks: AH authenticates but does not encrypt. ESP therefore protects data confidentiality across untrusted networks while still offering integrity and anti-replay protection. Because ESP can authenticate without including the outer IP header in the integrity check, it traverses NAT (especially with NAT-Traversal/UDP encapsulation) more readily than AH, which is why ESP dominates real-world IPsec VPN deployments.

For example, two branch offices connect over the internet with a site-to-site IPsec VPN using ESP in tunnel mode. Each office's gateway encrypts the entire IP packet leaving the local network, encapsulates it in a new ESP-protected packet, and sends it to the peer gateway, which decrypts and verifies it before forwarding to the internal host. Anyone intercepting the traffic sees only encrypted ESP packets, with internal addresses and data hidden and any alteration detected.