Types of Overlays
Network overlays can be implemented at different layers of the OSI model. There are various approaches to creating an overlay network, but a key characteristic of all of them is that they enable communication that is not strictly hardware bound.
Although overlay networks allow alternative traffic processing and forwarding, they still require a hardware layer for operation. A good example is the Cisco ACI infrastructure. Cisco ACI uses a CLOS hardware topology of devices that can process both Layer 2 and Layer 3 traffic. The physical cabling links enable traffic forwarding, but the network overlay determines which path through the network will be chosen.
Cisco ACI is not just a network overlay but a fully fledged Software-Defined Networking (SDN) system. It handles path selection automatically based on the policies that are applied to the APIC controller. Because this traffic forwarding does not rely on Layer 2 switching as a traditional switched network would, loop prevention mechanisms such as STP are not required inside the ACI overlay.
Depending on the connectivity requirements and network characteristics, various overlay types can be used. The choice of overlay also depends on the protocol support of the devices that are used to create it. For example, Cisco ACI is not supported on all switches in the Cisco Nexus lineup.
There are two types of network overlays: Layer 2 overlays and Layer 3 overlays. You can choose the appropriate option based on your goals and specific use case. The figure shows an example of a Layer 2 overlay on the left and a Layer 3 overlay on the right.
Overlays often encapsulate lower-layer traffic into higher-layer traffic, creating a virtual network.
Layer 2 overlay
As shown in the image, Layer 2 overlays emulate a Layer 2 LAN segment. The forwarding process uses Ethernet frame headers, allowing the transport of both IP and non-IP packets. The overlay creates a single subnet (a single Layer 2 domain). Layer 2 overlays are useful in environments where workload mobility is necessary. Also, they can emulate specific physical topologies. However, a key disadvantage of Layer 2 overlays is that they expand the Layer 2 domain, increasing the size of the flooding domain.
Layer 3 Overlay
Layer 3 overlays are beneficial when transporting IP packets while abstracting the underlying IP infrastructure. They are particularly useful for enabling IP mobility. Also, Layer 3 overlays can be used to carry IPv4 traffic over an IPv6 infrastructure or vice versa.
| Overlay | Protocol | Common Use Case |
|---|---|---|
| VXLAN | MAC-in-UDP | Cisco, VMware, Containers |
| NVGRE | GRE over IP | Cisco, Microsoft, Hyper-V |
| Geneve | UDP | Cisco, VMware, AWS |
Different commonly used overlay protocols provide different features, with VXLAN being the most used, especially in multi-tenant and diverse hardware environments.
Although Cisco devices support multiple overlay protocols, VXLAN remains one of the main protocols used. VXLAN provides a way to transfer generic Layer 2 traffic over Layer 3 networks, has excellent performance, and is well supported in hardware for hardware offloading.
You must understand VXLAN overlays to a significant degree when designing modern data center networks, especially if you want to implement microsegmentation for microservices such as in containerized environments. Kubernetes, a core containerization system in modern designs, has good support for VXLAN through use of Flannel solution for Container Network Interface (CNI).