MPLS VPN over IP Tunnels

MPLS VPN over IP Tunnels

RFC 5512 specifies a way for BGP to tell other BGP Speakers about the type of encapsulation to be used and its related information. BGP uses a new defined SAFI (Subsequent Address Family Identifier) value called Encapsulation SAFI. It can be used with AFI for IPv4 (AFI=1) or IPv6 (AFI=2). The AFI value to be used depends upon the IP network over which the encapsulated packet would be sent.

In a given BGP UPDATE message, the NLRI of Encapsulation SAFI consists of the IP address of the originator of the update. The encapsulation information, like the encapsulation protocol, is specified in the BGP Tunnel Encapsulation attribute.

RFC 5512 defines SAFI value 7 for Encapsulation SAFI (Cisco uses SAFI = 64). When a BGP Speaker advertises an Encapsulation NLRI via BGP, it uses its own address as the BGP Next-hop in the MP_REACH_NLRI or MP_UNREACH_NLRI attribute. The receiving router performs a recursive route lookup on this BGP next-hop.

BGP Tunnel Encapsulation attribute

The Tunnel Encapsulation attribute (Cisco calls it SAFI Specific Attribute SSA) is an optional transitive attribute which is composed of a set of TLVs. Each TLV contains information of a particular Tunnel.

The following packet capture shows a BGP UPDATE message carrying Tunnel information.

MPLS VPN over Multipoint L2TPv3 Tunnels

The L2TPv3 multipoint tunnel network allows Layer 3 VPN services to be carried through the IP core without the configuration of MPLS. A full-mesh topology is created between PE routers but only one tunnel is configured on each PE router. VPNv4 traffic is routed through this tunnel.

Configuring a Multipoint L2TPv3 Tunnel

An L2TPv3 Multipoint tunnel allows L3VPN service to be carried through the IP Core network, without the configuration of MPLS. L2TPv3 Multipoint tunnel allows multiple tunnel endpoints, which creates full-mesh topology between PE routers and hence requires only one tunnel. A special VRF L3VPN_L2TPv3 is configured on each PE router for the multipoint L2TPv3 tunnel which will carry L3VPN traffic between PE routers.

Tunnel Configuration on PE1

ip vrf L3VPN_L2TPv3                ! VRF for L2TPv3 Multipoint tunnel
 rd 1:1
!
ip vrf Customer_A                  ! VRF for the Customer
 rd 100:1
 route-target both 100:1
!
interface Loopback 0
 ip address 1.1.1.1 255.255.255.255
 ip ospf 1 area 0
!
interface Serial 2/0
 ip vrf forwarding Customer_A
 ip address 192.168.1.1 255.255.255.0
!
interface Tunnel 1
 ip vrf forwarding L3VPN_L2TPv3
 ip address 10.10.10.1 255.255.255.255
 tunnel source Loopback 0
 tunnel mode l3vpn l2tpv3 multipoint
!

Tunnel Configuration on PE2

ip vrf L3VPN_L2TPv3               
 rd 1:1
!
ip vrf Customer_A                 
 rd 100:1
 route-target both 100:1
!
interface Loopback 0
 ip address 2.2.2.2 255.255.255.255
 ip ospf 1 area 0
!
interface Serial 2/0
 ip vrf forwarding Customer_A
 ip address 192.168.2.1 255.255.255.0
!
interface Tunnel 1
 ip vrf forwarding L3VPN_L2TPv3
 ip address 10.10.10.2 255.255.255.255
 tunnel source Loopback 0
 tunnel mode l3vpn l2tpv3 multipoint
!

Tunnel Configuration on PE3

ip vrf L3VPN_L2TPv3               
 rd 1:1
!
ip vrf Customer_A                 
 rd 100:1
 route-target both 100:1
!
interface Loopback 0
 ip address 3.3.3.3 255.255.255.255
 ip ospf 1 area 0
!
interface Serial 2/0
 ip vrf forwarding Customer_A
 ip address 192.168.3.1 255.255.255.0
!
interface Tunnel 1
 ip vrf forwarding L3VPN_L2TPv3
 ip address 10.10.10.3 255.255.255.255
 tunnel source Loopback 0
 tunnel mode l3vpn l2tpv3 multipoint
!

Configuring BGP to advertise Tunnel attributes

BGP is used to advertise the tunnel type, tunnel capabilities and tunnel-specific attributes like Cookie ID and Session ID. BGP is also used to advertise VPNv4 routing information between PE routers. The next-hop advertised in BGP Update messages, triggers tunnel endpoint discovery.

BGP Configuration on PE1

router bgp 100
 neighbor 2.2.2.2 remote-as 100
 neighbor 2.2.2.2 update-source Loopback 0
 neighbor 3.3.3.3 remote-as 100
 neighbor 3.3.3.3 update-source Loopback 0
 !
 address-family vpnv4
 neighbor 2.2.2.2 activate
 neighbor 3.3.3.3 activate
 !
 address-family ipv4 tunnel                ! This causes BGP to advertise the tunnel endpoints and SAFI specific attributes (like tunnel type and tunnel capabilities) between PE routers.
 neighbor 2.2.2.2 activate
 neighbor 3.3.3.3 activate
 !
 address-family ipv4 vrf Customer_A
 redistribute connected
 exit-address-family
!

BGP Configuration on PE2

router bgp 100

neighbor 1.1.1.1 remote-as 100

 neighbor 1.1.1.1 update-source Loopback 0
 neighbor 3.3.3.3 remote-as 100
 neighbor 3.3.3.3 update-source Loopback 0

!

 address-family vpnv4
 neighbor 1.1.1.1 activate
 neighbor 3.3.3.3 activate
 !
 address-family ipv4 tunnel               
 neighbor 1.1.1.1 activate
 neighbor 3.3.3.3 activate
 !
 address-family ipv4 vrf Customer_A
 redistribute connected
 exit-address-family
!

BGP Configuration on PE3

router bgp 100

neighbor 1.1.1.1 remote-as 100

 neighbor 1.1.1.1 update-source Loopback 0
 neighbor 2.2.2.2 remote-as 100
 neighbor 2.2.2.2 update-source Loopback 0

!

 address-family vpnv4
 neighbor 1.1.1.1 activate
 neighbor 2.2.2.2 activate
 !
 address-family ipv4 tunnel               
 neighbor 1.1.1.1 activate
 neighbor 2.2.2.2 activate
 !
 address-family ipv4 vrf Customer_A
 redistribute connected
 exit-address-family
!

Configuring a Static Default route pointing to Tunnel interface and BGP to resolve the next-hop in the tunnel VRF

By default, BGP will try to resolve the next-hop in the global routing table which will send the VPN packet directly to the P-router without the L2TPv3 header/encapsulation causing the P-router to drop the packet as they don't have VPN information in their routing table.

To avoid this, a route-map is configured to resolve the next-hop in the tunnel VRF. The static default route pointing to the tunnel interface will encapsulate the VPN packets with source address as the local loopback address and destination as the remote-PE loopback address, and forward to the P routers.

PE1 router

ip route vrf L3VPN_L2TPv3 0.0.0.0 0.0.0.0 Tunnel 1
!
route-map NEXTHOP_LOOKUP permit 10
 set ip next-hop in-vrf L3VPN_L2TPv3
!
router bgp 100
 address-family vpnv4
 neighbor 2.2.2.2 route-map NEXTHOP_LOOKUP in
 neighbor 3.3.3.3 route-map NEXTHOP_LOOKUP in
!

PE2 router

ip route vrf L3VPN_L2TPv3 0.0.0.0 0.0.0.0 Tunnel 1
!
route-map NEXTHOP_LOOKUP permit 10
 set ip next-hop in-vrf L3VPN_L2TPv3
!
router bgp 100
 address-family vpnv4
 neighbor 1.1.1.1 route-map NEXTHOP_LOOKUP in
 neighbor 3.3.3.3 route-map NEXTHOP_LOOKUP in
!

PE3 router

ip route vrf L3VPN_L2TPv3 0.0.0.0 0.0.0.0 Tunnel 1
!
route-map NEXTHOP_LOOKUP permit 10
 set ip next-hop in-vrf L3VPN_L2TPv3
!
router bgp 100
 address-family vpnv4
 neighbor 1.1.1.1 route-map NEXTHOP_LOOKUP in
 neighbor 2.2.2.2 route-map NEXTHOP_LOOKUP in
!

Verification

When BGP forms neighborship, it exchanges BGP capabilities using UPDATE message. Here, PE routers notify remote PE routers of Tunnel SAFI capability using SAFI = 64.

debug ip bgp

PE1#
00:04:48: BGP: 2.2.2.2 passive open to 1.1.1.1
00:04:48: BGP: 2.2.2.2 went from Active to Idle
00:04:48: BGP: 2.2.2.2 went from Idle to Connect
00:04:48: BGP: 2.2.2.2 rcv message type 1, length (excl. header) 50
00:04:48: BGP: 2.2.2.2 rcv OPEN, version 4, holdtime 180 seconds
00:04:48: BGP: 2.2.2.2 went from Connect to OpenSent
00:04:48: BGP: 2.2.2.2 sending OPEN, version 4, my as: 100, holdtime 180 seconds
00:04:48: BGP: 2.2.2.2 rcv OPEN w/ OPTION prameter len: 40
00:04:48: BGP: 2.2.2.2 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
00:04:48: BGP: 2.2.2.2 OPEN has CAPABILITY code: 1, length 4
00:04:48: BGP: 2.2.2.2 OPEN has MP_EXT CAP for afi/safi: 1/1
00:04:48: BGP: 2.2.2.2 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
00:04:48: BGP: 2.2.2.2 OPEN has CAPABILITY code: 1, length 4
00:04:48: BGP: 2.2.2.2 OPEN has MP_EXT CAP for afi/safi: 1/64
00:04:48: BGP: 2.2.2.2 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
00:04:48: BGP: 2.2.2.2 OPEN has CAPABILITY code: 1, length 4
00:04:48: BGP: 2.2.2.2 OPEN has MP_EXT CAP for afi/safi: 1/128
00:04:48: BGP: 2.2.2.2 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
00:04:48: BGP: 2.2.2.2 OPEN has CAPABILITY code: 128, length 0
00:04:48: BGP: 2.2.2.2 OPEN has ROUTE-REFRESH capability(old) for all address-families
00:04:48: BGP: 2.2.2.2 rcvd OPEN w/ optional parameter type 2 (Capability) len 2
00:04:48: BGP: 2.2.2.2 OPEN has CAPABILITY code: 2, length 0
00:04:48: BGP: 2.2.2.2 OPEN has ROUTE-REFRESH capability for all address-families
00:04:48: BGP: 2.2.2.2 rcvd OPEN w/ optional parameter type 2 (Capability) len 6
00:04:48: BGP: 2.2.2.2 OPEN has CAPABILITY code: 65, length 4
00:04:48: BGP: 2.2.2.2 OPEN has 4-byte ASN CAP for: 100
BGP: 2.2.2.2 rcvd OPEN w/ remote AS 100, 4-byte remote AS 100
00:04:48: BGP: 2.2.2.2 went from OpenSent to OpenConfirm
00:04:48: BGP: 2.2.2.2 send message type 1, length (incl. header) 69
00:04:48: BGP: 2.2.2.2 went from OpenConfirm to Established
00:04:48: %BGP-5-ADJCHANGE: neighbor 2.2.2.2 Up

The BGP UPDATE message also carries tunnel endpoint. After receiving Tunnel SAFI, the PE router initiates tunnel discovery.

Tunnel Endpoint

PE1# show tunnel endpoint
 Tunnel1 running in Multi-L2TPv3 (L3VPN) mode
  RFC2547/L3VPN Tunnel endpoint discovery is active on Tu1
  Transporting l3vpn traffic to all routes recursing through "L3VPN_L2TPv3"
 Endpoint 2.2.2.2 via destination 2.2.2.2
  Session 1025, High Cookie 0x4BB817DC Low Cookie 0xEFDA795B
 Endpoint 3.3.3.3 via destination 3.3.3.3
  Session 1025, High Cookie 0x6E6FD384 Low Cookie 0xAD6A4E26
 Tunnel Endpoint Process Active
 MGRE L3VPN Summary
   Active Tunnel: None
 L2tpv3 L3VPN Summary
   Active Tunnel: Tunnel1 Current receive Session 1025,
                              High Cookie 0xA61F7D20 Low Cookie 0xAF080CCD
   L2TPv3 cookie mismatch counters: 0

The show ip bgp vpnv4 command shows that the route-map has worked and the next-hop is showing to be resolved in the Tunnel VRF.

show ip bgp vpnv4

PE1# show ip bgp vpnv4 all 192.168.2.0
BGP routing table entry for 100:1:192.168.2.0/24, version 5
Paths: (1 available, best #1, table Customer_A)
  Not advertised to any peer
  Local
    2.2.2.2 (metric 3) (via L3VPN_L2TPv3) from 2.2.2.2 (2.2.2.2)
      Origin incomplete, metric 0, localpref 100, valid, internal, best
      Extended Community: RT:100:1

mpls labels in/out nolabel/35 ! Label advertised by PE2 router for 192.168.2.0/24 prefix

PE1# show ip bgp vpnv4 all 192.168.3.0
BGP routing table entry for 100:1:192.168.3.0/24, version 7
Paths: (1 available, best #1, table Customer_A)
  Not advertised to any peer
  Local
    3.3.3.3 (metric 3) (via L3VPN_L2TPv3) from 3.3.3.3 (3.3.3.3)
      Origin incomplete, metric 0, localpref 100, valid, internal, best
      Extended Community: RT:100:1

mpls labels in/out nolabel/36 ! Label advertised by PE3 router for 192.168.3.0/24 prefix

PE1# show ip route vrf Customer_A | begin Gateway
Gateway of last resort is not set
C    192.168.1.0/24 is directly connected, Serial2/0

B 192.168.2.0/24 [200/0] via 2.2.2.2 (L3VPN_L2TPv3), 00:17:07 ! The VRF name in the parenthesis indicates the next-hop resolved in Tunnel VRF

B    192.168.3.0/24 [200/0] via 3.3.3.3 (L3VPN_L2TPv3), 00:07:21

The show ip bgp ipv4 tunnel command shows tunnel specific information.

show ip bgp ipv4 tunnel

PE1# show ip bgp ipv4 tunnel 2.2.2.2
BGP routing table entry for 2.2.2.2/32, version 3
Paths: (1 available, best #1, table IPv4-Tunnel-BGP-Table)
  Not advertised to any peer
  Local
    2.2.2.2 (metric 3) from 2.2.2.2 (2.2.2.2)
      Origin incomplete, metric 0, localpref 100, valid, internal, best
      SAFI Specific Attribute: ssacount=1
       type L2TP (transitive), len 16
        pref 0,flags 0x0,session 1025,cookielen 8,cookie 0x4BB817DC 0xEFDA795B
PE1# show ip bgp ipv4 tunnel 3.3.3.3
BGP routing table entry for 3.3.3.3/32, version 4
Paths: (1 available, best #1, table IPv4-Tunnel-BGP-Table)
  Not advertised to any peer
  Local
    3.3.3.3 (metric 3) from 3.3.3.3 (3.3.3.3)
      Origin incomplete, metric 0, localpref 100, valid, internal, best
      SAFI Specific Attribute: ssacount=1
       type L2TP (transitive), len 16
        pref 0,flags 0x0,session 1025,cookielen 8,cookie 0x6E6FD384 0xAD6A4E26

Further reading:

  1. http://mpls-configuration-on-cisco-ios-software.org.ua/1587051990/ch10lev1sec5.html
  2. http://www.cisco.com/en/US/docs/ios/12_0s/feature/guide/csgl3vpn.pdf
  3. http://tools.ietf.org/id/draft-townsley-l3vpn-l2tpv3-01.txt
  4. http://tools.ietf.org/pdf/rfc5512.pdf