MPLS VPN over IP Tunnels
MPLS VPN over IP Tunnels
RFC 5512 specifies a way for BGP to tell other BGP Speakers about the type of encapsulation to be used and its related information. BGP uses a new defined SAFI (Subsequent Address Family Identifier) value called Encapsulation SAFI. It can be used with AFI for IPv4 (AFI=1) or IPv6 (AFI=2). The AFI value to be used depends upon the IP network over which the encapsulated packet would be sent.
In a given BGP UPDATE message, the NLRI of Encapsulation SAFI consists of the IP address of the originator of the update. The encapsulation information, like the encapsulation protocol, is specified in the BGP Tunnel Encapsulation attribute.
RFC 5512 defines SAFI value 7 for Encapsulation SAFI (Cisco uses SAFI = 64). When a BGP Speaker advertises an Encapsulation NLRI via BGP, it uses its own address as the BGP Next-hop in the MP_REACH_NLRI or MP_UNREACH_NLRI attribute. The receiving router performs a recursive route lookup on this BGP next-hop.
BGP Tunnel Encapsulation attribute
The Tunnel Encapsulation attribute (Cisco calls it SAFI Specific Attribute SSA) is an optional transitive attribute which is composed of a set of TLVs. Each TLV contains information of a particular Tunnel.
The following packet capture shows a BGP UPDATE message carrying Tunnel information.
MPLS VPN over Multipoint L2TPv3 Tunnels
The L2TPv3 multipoint tunnel network allows Layer 3 VPN services to be carried through the IP core without the configuration of MPLS. A full-mesh topology is created between PE routers but only one tunnel is configured on each PE router. VPNv4 traffic is routed through this tunnel.
Configuring a Multipoint L2TPv3 Tunnel
An L2TPv3 Multipoint tunnel allows L3VPN service to be carried through the IP Core network, without the configuration of MPLS. L2TPv3 Multipoint tunnel allows multiple tunnel endpoints, which creates full-mesh topology between PE routers and hence requires only one tunnel. A special VRF L3VPN_L2TPv3 is configured on each PE router for the multipoint L2TPv3 tunnel which will carry L3VPN traffic between PE routers.
Tunnel Configuration on PE1
ip vrf L3VPN_L2TPv3 ! VRF for L2TPv3 Multipoint tunnel
rd 1:1
!
ip vrf Customer_A ! VRF for the Customer
rd 100:1
route-target both 100:1
!
interface Loopback 0
ip address 1.1.1.1 255.255.255.255
ip ospf 1 area 0
!
interface Serial 2/0
ip vrf forwarding Customer_A
ip address 192.168.1.1 255.255.255.0
!
interface Tunnel 1
ip vrf forwarding L3VPN_L2TPv3
ip address 10.10.10.1 255.255.255.255
tunnel source Loopback 0
tunnel mode l3vpn l2tpv3 multipoint
!
Tunnel Configuration on PE2
ip vrf L3VPN_L2TPv3
rd 1:1
!
ip vrf Customer_A
rd 100:1
route-target both 100:1
!
interface Loopback 0
ip address 2.2.2.2 255.255.255.255
ip ospf 1 area 0
!
interface Serial 2/0
ip vrf forwarding Customer_A
ip address 192.168.2.1 255.255.255.0
!
interface Tunnel 1
ip vrf forwarding L3VPN_L2TPv3
ip address 10.10.10.2 255.255.255.255
tunnel source Loopback 0
tunnel mode l3vpn l2tpv3 multipoint
!
Tunnel Configuration on PE3
ip vrf L3VPN_L2TPv3
rd 1:1
!
ip vrf Customer_A
rd 100:1
route-target both 100:1
!
interface Loopback 0
ip address 3.3.3.3 255.255.255.255
ip ospf 1 area 0
!
interface Serial 2/0
ip vrf forwarding Customer_A
ip address 192.168.3.1 255.255.255.0
!
interface Tunnel 1
ip vrf forwarding L3VPN_L2TPv3
ip address 10.10.10.3 255.255.255.255
tunnel source Loopback 0
tunnel mode l3vpn l2tpv3 multipoint
!
Configuring BGP to advertise Tunnel attributes
BGP is used to advertise the tunnel type, tunnel capabilities and tunnel-specific attributes like Cookie ID and Session ID. BGP is also used to advertise VPNv4 routing information between PE routers. The next-hop advertised in BGP Update messages, triggers tunnel endpoint discovery.
BGP Configuration on PE1
router bgp 100
neighbor 2.2.2.2 remote-as 100
neighbor 2.2.2.2 update-source Loopback 0
neighbor 3.3.3.3 remote-as 100
neighbor 3.3.3.3 update-source Loopback 0
!
address-family vpnv4
neighbor 2.2.2.2 activate
neighbor 3.3.3.3 activate
!
address-family ipv4 tunnel ! This causes BGP to advertise the tunnel endpoints and SAFI specific attributes (like tunnel type and tunnel capabilities) between PE routers.
neighbor 2.2.2.2 activate
neighbor 3.3.3.3 activate
!
address-family ipv4 vrf Customer_A
redistribute connected
exit-address-family
!
BGP Configuration on PE2
router bgp 100
neighbor 1.1.1.1 remote-as 100
neighbor 1.1.1.1 update-source Loopback 0
neighbor 3.3.3.3 remote-as 100
neighbor 3.3.3.3 update-source Loopback 0
!
address-family vpnv4
neighbor 1.1.1.1 activate
neighbor 3.3.3.3 activate
!
address-family ipv4 tunnel
neighbor 1.1.1.1 activate
neighbor 3.3.3.3 activate
!
address-family ipv4 vrf Customer_A
redistribute connected
exit-address-family
!
BGP Configuration on PE3
router bgp 100
neighbor 1.1.1.1 remote-as 100
neighbor 1.1.1.1 update-source Loopback 0
neighbor 2.2.2.2 remote-as 100
neighbor 2.2.2.2 update-source Loopback 0
!
address-family vpnv4
neighbor 1.1.1.1 activate
neighbor 2.2.2.2 activate
!
address-family ipv4 tunnel
neighbor 1.1.1.1 activate
neighbor 2.2.2.2 activate
!
address-family ipv4 vrf Customer_A
redistribute connected
exit-address-family
!
Configuring a Static Default route pointing to Tunnel interface and BGP to resolve the next-hop in the tunnel VRF
By default, BGP will try to resolve the next-hop in the global routing table which will send the VPN packet directly to the P-router without the L2TPv3 header/encapsulation causing the P-router to drop the packet as they don't have VPN information in their routing table.
To avoid this, a route-map is configured to resolve the next-hop in the tunnel VRF. The static default route pointing to the tunnel interface will encapsulate the VPN packets with source address as the local loopback address and destination as the remote-PE loopback address, and forward to the P routers.
PE1 router
ip route vrf L3VPN_L2TPv3 0.0.0.0 0.0.0.0 Tunnel 1
!
route-map NEXTHOP_LOOKUP permit 10
set ip next-hop in-vrf L3VPN_L2TPv3
!
router bgp 100
address-family vpnv4
neighbor 2.2.2.2 route-map NEXTHOP_LOOKUP in
neighbor 3.3.3.3 route-map NEXTHOP_LOOKUP in
!
PE2 router
ip route vrf L3VPN_L2TPv3 0.0.0.0 0.0.0.0 Tunnel 1
!
route-map NEXTHOP_LOOKUP permit 10
set ip next-hop in-vrf L3VPN_L2TPv3
!
router bgp 100
address-family vpnv4
neighbor 1.1.1.1 route-map NEXTHOP_LOOKUP in
neighbor 3.3.3.3 route-map NEXTHOP_LOOKUP in
!
PE3 router
ip route vrf L3VPN_L2TPv3 0.0.0.0 0.0.0.0 Tunnel 1
!
route-map NEXTHOP_LOOKUP permit 10
set ip next-hop in-vrf L3VPN_L2TPv3
!
router bgp 100
address-family vpnv4
neighbor 1.1.1.1 route-map NEXTHOP_LOOKUP in
neighbor 2.2.2.2 route-map NEXTHOP_LOOKUP in
!
Verification
When BGP forms neighborship, it exchanges BGP capabilities using UPDATE message. Here, PE routers notify remote PE routers of Tunnel SAFI capability using SAFI = 64.
debug ip bgp
PE1#
00:04:48: BGP: 2.2.2.2 passive open to 1.1.1.1
00:04:48: BGP: 2.2.2.2 went from Active to Idle
00:04:48: BGP: 2.2.2.2 went from Idle to Connect
00:04:48: BGP: 2.2.2.2 rcv message type 1, length (excl. header) 50
00:04:48: BGP: 2.2.2.2 rcv OPEN, version 4, holdtime 180 seconds
00:04:48: BGP: 2.2.2.2 went from Connect to OpenSent
00:04:48: BGP: 2.2.2.2 sending OPEN, version 4, my as: 100, holdtime 180 seconds
00:04:48: BGP: 2.2.2.2 rcv OPEN w/ OPTION prameter len: 40
00:04:48: BGP: 2.2.2.2 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
00:04:48: BGP: 2.2.2.2 OPEN has CAPABILITY code: 1, length 4
00:04:48: BGP: 2.2.2.2 OPEN has MP_EXT CAP for afi/safi: 1/1
00:04:48: BGP: 2.2.2.2 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
00:04:48: BGP: 2.2.2.2 OPEN has CAPABILITY code: 1, length 4
00:04:48: BGP: 2.2.2.2 OPEN has MP_EXT CAP for afi/safi: 1/64
00:04:48: BGP: 2.2.2.2 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
00:04:48: BGP: 2.2.2.2 OPEN has CAPABILITY code: 1, length 4
00:04:48: BGP: 2.2.2.2 OPEN has MP_EXT CAP for afi/safi: 1/128
00:04:48: BGP: 2.2.2.2 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
00:04:48: BGP: 2.2.2.2 OPEN has CAPABILITY code: 128, length 0
00:04:48: BGP: 2.2.2.2 OPEN has ROUTE-REFRESH capability(old) for all address-families
00:04:48: BGP: 2.2.2.2 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
00:04:48: BGP: 2.2.2.2 OPEN has CAPABILITY code: 2, length 0
00:04:48: BGP: 2.2.2.2 OPEN has ROUTE-REFRESH capability for all address-families
00:04:48: BGP: 2.2.2.2 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
00:04:48: BGP: 2.2.2.2 OPEN has CAPABILITY code: 65, length 4
00:04:48: BGP: 2.2.2.2 OPEN has 4-byte ASN CAP for: 100
BGP: 2.2.2.2 rcvd OPEN w/ remote AS 100, 4-byte remote AS 100
00:04:48: BGP: 2.2.2.2 went from OpenSent to OpenConfirm
00:04:48: BGP: 2.2.2.2 send message type 1, length (incl. header) 69
00:04:48: BGP: 2.2.2.2 went from OpenConfirm to Established
00:04:48: %BGP-5-ADJCHANGE: neighbor 2.2.2.2 Up
The BGP UPDATE message also carries tunnel endpoint. After receiving Tunnel SAFI, the PE router initiates tunnel discovery.
Tunnel Endpoint
PE1# show tunnel endpoint
Tunnel1 running in Multi-L2TPv3 (L3VPN) mode
RFC2547/L3VPN Tunnel endpoint discovery is active on Tu1
Transporting l3vpn traffic to all routes recursing through "L3VPN_L2TPv3"
Endpoint 2.2.2.2 via destination 2.2.2.2
Session 1025, High Cookie 0x4BB817DC Low Cookie 0xEFDA795B
Endpoint 3.3.3.3 via destination 3.3.3.3
Session 1025, High Cookie 0x6E6FD384 Low Cookie 0xAD6A4E26
Tunnel Endpoint Process Active
MGRE L3VPN Summary
Active Tunnel: None
L2tpv3 L3VPN Summary
Active Tunnel: Tunnel1 Current receive Session 1025,
High Cookie 0xA61F7D20 Low Cookie 0xAF080CCD
L2TPv3 cookie mismatch counters: 0
The show ip bgp vpnv4 command shows that the route-map has worked and the next-hop is showing to be resolved in the Tunnel VRF.
show ip bgp vpnv4
PE1# show ip bgp vpnv4 all 192.168.2.0
BGP routing table entry for 100:1:192.168.2.0/24, version 5
Paths: (1 available, best #1, table Customer_A)
Not advertised to any peer
Local
2.2.2.2 (metric 3) (via L3VPN_L2TPv3) from 2.2.2.2 (2.2.2.2)
Origin incomplete, metric 0, localpref 100, valid, internal, best
Extended Community: RT:100:1
mpls labels in/out nolabel/35
! Label advertised by PE2 router for 192.168.2.0/24 prefix
PE1# show ip bgp vpnv4 all 192.168.3.0
BGP routing table entry for 100:1:192.168.3.0/24, version 7
Paths: (1 available, best #1, table Customer_A)
Not advertised to any peer
Local
3.3.3.3 (metric 3) (via L3VPN_L2TPv3) from 3.3.3.3 (3.3.3.3)
Origin incomplete, metric 0, localpref 100, valid, internal, best
Extended Community: RT:100:1
mpls labels in/out nolabel/36
! Label advertised by PE3 router for 192.168.3.0/24 prefix
PE1# show ip route vrf Customer_A | begin Gateway
Gateway of last resort is not set
C 192.168.1.0/24 is directly connected, Serial2/0
B 192.168.2.0/24 [200/0] via 2.2.2.2 (L3VPN_L2TPv3), 00:17:07
! The VRF name in the parenthesis indicates the next-hop resolved in Tunnel VRF
B 192.168.3.0/24 [200/0] via 3.3.3.3 (L3VPN_L2TPv3), 00:07:21
The show ip bgp ipv4 tunnel command shows tunnel specific information.
show ip bgp ipv4 tunnel
PE1# show ip bgp ipv4 tunnel 2.2.2.2
BGP routing table entry for 2.2.2.2/32, version 3
Paths: (1 available, best #1, table IPv4-Tunnel-BGP-Table)
Not advertised to any peer
Local
2.2.2.2 (metric 3) from 2.2.2.2 (2.2.2.2)
Origin incomplete, metric 0, localpref 100, valid, internal, best
SAFI Specific Attribute: ssacount=1
type L2TP (transitive), len 16
pref 0,flags 0x0,session 1025,cookielen 8,cookie 0x4BB817DC 0xEFDA795B
PE1# show ip bgp ipv4 tunnel 3.3.3.3
BGP routing table entry for 3.3.3.3/32, version 4
Paths: (1 available, best #1, table IPv4-Tunnel-BGP-Table)
Not advertised to any peer
Local
3.3.3.3 (metric 3) from 3.3.3.3 (3.3.3.3)
Origin incomplete, metric 0, localpref 100, valid, internal, best
SAFI Specific Attribute: ssacount=1
type L2TP (transitive), len 16
pref 0,flags 0x0,session 1025,cookielen 8,cookie 0x6E6FD384 0xAD6A4E26
Further reading: