OTV Use Case
In this article, I will demonstrate a use case of Overlay Transport Virtualization (OTV) technology from Cisco. As the name suggests, it is an overlay technology which is used to extend Layer 2 networks, usually between data centers. The current implementation is only supported on Cisco Nexus 7000 switches, NX-OS 5.1 and above. In NX-OS 5.1, OTV requires IP Multicast support in the transport network but the recent release also allows Unicast transport support. Together with the use case, I will try to go over the concepts of OTV. Let's set the stage first.
In the figure above, there are 3 types of servers in each data center connected with each other via 4 DCI routers that are of interest to us. The requirement is as follows-
It is clear from the requirements that the servers do not need SVI for VLANs 5 and 6 which removes some complexity associated with FHRP. The management of the servers is via VLAN 10.
OTV Terminology
First some OTV terminology -
Control Plane
Inherently, OTV uses IS-IS protocol to exchange MAC address reachability information between Edge devices (no IS-IS knowledge is required for OTV as such). However, before Edge devices can exchange MAC address information, they need to form peer relationship. This adjacency can be formed in 2 ways, depending on the type of transport network - Multicast enabled (discussed next) and Unicast enabled (not discussed in this article).
Multicast Enabled Transport Network
If the transport network is multicast enabled, a specific multicast group can be used by OTV Edge devices to exchange control protocol messages. All OTV Edge devices are configured to join a specific Any Source Multicast (ASM) group. ASM group is where anyone can be a source and a receiver for the group. The multicast transport network is recommended to be based on PIM Bi-directional, however, PIM Sparse-mode (PIM-SM) with static Rendezvous Point (RP) should work equally fine.
The same process occurs in the other direction and results in the OTV control-plane adjacencies between OTV Edge devices. The exchange of MAC address information is now possible. The MAC address advertisement process is quite similar to above -
The MAC address information is carried in IS-IS Type-Length-Value (TLV). The MAC address information ages out in 30 minutes by default.
Data Plane : Unicast Traffic
Once OTV adjacencies are formed and MAC address information is exchanged by OTV Edge devices, traffic can flow across the overlay network. If App Server 1 wants to communicate with DB Server 1 (i.e. within the same site), the Layer2 switch looks up the CAM table and it points to a local interface. The switch can now perform traditional switching.
However, if EMS Server 1 in left data center wants to communicate with EMS Server 2 in right data center, traffic needs to flow over the overlay network.
Data Plane : Multicast Traffic
Although it is not relevant to this use case, it is possible that a host/server in one data center is a multicast-source and another host/server is a multicast-receiver in another data center for the same Layer2 segment. Let's consider this scenario - EMS Server 1 is a multicast-source.
Multicast-enabled Transport Network
Cisco recommends to enable Source Specific Multicast (SSM) group range in the transport network to carry this Layer2 multicast traffic between sites over the overlay network. This group range is specified using Data-Group (GD) in OTV and is different to ASM group (OTV Control-Group).
Data Plane : Broadcast Traffic
STP Isolation
By default, Spanning Tree Protocol (STP) BPDUs are not allowed across the overlay by OTV Edge devices. Hence, every site is an independent STP domain
Unknown Unicast Handling
In the current release of NX-OS, the OTV Edge device does not flood any unknown unicast frames across the overlay. However, like in traditional switches, the unknown unicast traffic is still flooded out the regular Ethernet interfaces.
ARP Optimization
ARP Request (Layer2 broadcast frame) is allowed across the overlay network to all remote sites. ARP Reply (Layer2 unicast frame) is also allowed across the overlay network. Also, the OTV Edge devices snoop and cache the ARP Reply messages. Any subsequent ARP Requests for that MAC address is served by the OTV Edge devices rather than flooding across the overlay network. This requires that OTV ARP-timer be lower than MAC address aging-timer.
OTV Encapsulation
OTV adds a further 42 bytes on all packets traveling across the overlay network. The OTV Edge device removes the CRC and 802.1Q fields from the original Layer2 frame. It then adds an OTV Shim Header which includes this 802.1Q field (this includes the priority P-bit value) and the Overlay ID information. It also includes an external IP header for the transport network. All OTV packets have Don't Fragment (DF) bit set to 1 in the external IP header.
OTV Configuration
OTV requires Transport_Services_Package license installed on Nexus switches. The configuration steps are -
Step 1 : Enable OTV feature
feature otv
From NX-OS 5.2 (1) onwards, the OTV Site-ID configuration is mandatory and must match on all Edge devices belonging to the same site.
otv site-identifier 0x1
Step 2 : Create VLANs to be extended across the overlay
vlan 5
name EMS-VLAN
vlan 6
name App-DB-VLAN
Step 3 : Create Site VLAN
It is recommended that a dedicated VLAN be used as Site VLAN. It is not extended across the overlay.
vlan 15
name Site-VLAN
!
otv site-vlan 15
Step 4 : OTV Join Interface
Only one Join interface can be specified per overlay on each device.
interface Ethernet 1/1
description [ OTV Join Interface ]
no switchport
mtu 9000
ip address 10.1.1.1/30
igmp version 3
no shutdown
end
Step 5 : OTV Internal Interfaces
These are the Layer2 (trunk or access) interfaces on the OTV Edge devices.
interface Ethernet 2/1-2
description [ OTV Internal Interface ]
switchport
switchport mode trunk
switchport trunk allowed vlan 5,6,15
no shutdown
end
Step 6 : OTV Overlay Interface
The Overlay interface ID must match on all sites. Multiple overlay interfaces can be created. But, a VLAN can only be assigned to one overlay interface.
interface overlay 1
description [ OTV Overlay Interface ]
otv join-interface Ethernet 1/1
otv control-group 239.0.0.1
otv data-group 232.0.0.0/24
otv extended-vlan 5,6
no shutdown
end
There are some useful output verification commands to check the state of OTV Overlay interface and adjacency.
Conclusion
OTV does provide a way to extend Layer2 segments over Layer3 transport network. OTV configuration is very simple. Multi-homing and loop-prevention features are great. However, current implementation has certain limitations like -