BGP Multipath Load Sharing for Both eBGP and iBGP in an MPLS-VPN

BGP Multipath Load Sharing for Both eBGP and iBGP in an MPLS-VPN

A BGP routing process installs only a single path as the best path in the routing table by default. Also, BGP prefers eBGP paths over iBGP paths by default. The maximum-paths command allows BGP to install multiple paths in the routing table, however, it will still advertise only ONE BEST path to its eBGP neighbor. The multipath load-sharing for both eBGP and iBGP allows to configure multipath load-balancing with both external and internal paths.

Network topology:

CE1 and CE2 routers are two customer devices of different organizations, which are connected to the SP, offering an MPLS VPN service. CE2 is multihomed that is connected to two SP routers. There is an extranet-VPN connection between CE1 and CE2 routers.

Initial Configuration:

CE1 & CE2 configuration

CE1 router:
interface serial 0/0
 ip address 172.64.1.1 255.255.255.252
!
router bgp 65001
 neighbor 172.64.1.2 remote-as 100
 network 10.1.1.0 mask 255.255.255.0
!
CE2 router:
interface serial 0/0
 ip address 172.64.2.1 255.255.255.252
!
interface serial 0/1
 ip address 172.64.3.1 255.255.255.252
!
router bgp 65002
 neighbor 172.64.2.2 remote-as 100
 neighbor 172.64.3.2 remote-as 100
 network 10.2.2.0 mask 255.255.255.0
!

PE1 configuration

ip vrf CUST1
 rd 1:1
 route-target both 1:1
 route-target import 1:2
 route-target import 2:2
!
ip vrf CUST2
 rd 1:2
 route-target both 1:2
 route-target import 1:1
!
interface Loopback 0
 ip address 1.1.1.1 255.255.255.255
!
interface serial 0/0
 ip vrf forwarding CUST1
 ip address 172.64.1.2 255.255.255.252
!
interface serial 0/1
 ip vrf forwarding CUST2
 ip address 172.64.2.2 255.255.255.252
!
router bgp 100
 neighbor 2.2.2.2 remote-as 100
 neighbor 2.2.2.2 update-source Loopback 0
 !
 address-family vpnv4
 neighbor 2.2.2.2 activate
 neighbor 2.2.2.2 send-community both
 exit-address-family
 !
 address-family ipv4 vrf CUST1
 neighbor 172.64.1.1 remote-as 65001
 neighbor 172.64.1.1 activate
 redistribute connected
 exit-address-family
 !
 address-family ipv4 vrf CUST2
 neighbor 172.64.2.1 remote-as 65002
 neighbor 172.64.2.1 activate
 redistribute connected
 exit-address-family
!

PE2 configuration

ip vrf CUST2
 rd 2:2
 route-target both 2:2
 route-target import 1:1
!
interface Loopback 0
 ip address 2.2.2.2 255.255.255.255
!
interface serial 0/0
 ip vrf forwarding CUST2
 ip address 172.64.3.2 255.255.255.252
!
router bgp 100
 neighbor 1.1.1.1 remote-as 100
 neighbor 1.1.1.1 update-source Loopback 0
 !
 address-family vpnv4
 neighbor 1.1.1.1 activate
 neighbor 1.1.1.1 send-community both
 exit-address-family
 !
 address-family ipv4 vrf CUST2
 neighbor 172.64.3.1 remote-as 65002
 neighbor 172.64.3.1 activate
 redistribute connected
 exit-address-family
!

Default Behavior:

PE1 router receives 10.2.2.0/24 network from CE2 router over its eBGP connection imported from vrf CUST2. It installs this route as VPNv4 route in its VRF CUST1 BGP table as 1:2:10.2.2.0/24 with next-hop as CE2 router (172.64.2.1).

CE2 also advertises 10.2.2.0/24 to PE2 router over eBGP connection. PE2 router also installs this route as VPNv4 route in its VRF CUST1 BGP table as 2:2:10.2.2.0/24 with next-hop as PE2 router (2.2.2.2). Since there is an iBGP peering between PE1 and PE2 routers, PE2 router also advertises this VPNv4 route to PE1 router. PE1 router is configured to accept routes with RD 2:2. Hence, PE1 router will install this VPNv4 route in its BGP table.

However, by default, BGP will install only one route in the VRF routing table of CUST1.

10.2.2.0/24 on PE1

PE1# show ip bgp vpnv4 vrf CUST1 10.2.2.0
BGP routing table entry for 1:1:10.2.2.0/24, version 13
Paths: (2 available, best #1, table CUST1)
  Advertised to update-groups:
     2
  65002, imported path from 1:2:10.2.2.0/24
    172.64.2.1 from 172.64.2.1 (10.2.2.1)
      Origin IGP, metric 0, localpref 100, valid, external, best
      Extended Community: RT:1:2
  65002, imported path from 2:2:10.2.2.0/24
    2.2.2.2 (metric 11) from 2.2.2.2 (2.2.2.2)
      Origin IGP, metric 0, localpref 100, valid, internal
      Extended Community: RT:2:2
      mpls labels in/out nolabel/20
PE1# show ip route vrf CUST1
Routing Table: CUST1
---- output omitted ----
Gateway of last resort is not set
     172.64.0.0/30 is subnetted, 3 subnets
C       172.64.1.0 is directly connected, Serial0/0
B       172.64.2.0 is directly connected, 00:13:31, Serial0/1
B       172.64.3.0 [200/0] via 2.2.2.2, 00:06:45
     10.0.0.0/24 is subnetted, 2 subnets
B       10.2.2.0 [20/0] via 172.64.2.1 (CUST2), 00:13:31
B       10.1.1.0 [20/0] via 172.64.1.1, 00:13:33

PE1 router will advertise only one path to 10.2.2.0/24 to CE1 router.

Enabling Multipath feature:

The multipath feature can be enabled using maximum-paths command under address-family for a particular VRF in BGP. It allows to store multiple paths for a particular prefix. The maximum-paths eibgp <value> command allows multiple eBGP and iBGP paths in the BGP table for a prefix.

Enabling multipath feature on PE1 for vrf CUST1

router bgp 100
 !
 address-family ipv4 vrf CUST1
 maximum-paths eibgp 2
 exit-address-family
!

After the multipath feature is enabled, PE1 router will install both above mentioned routes in its routing table.

10.2.2.0/24 on PE1

PE1# show ip route vrf CUST1 10.2.2.0
Routing entry for 10.2.2.0/24
  Known via "bgp 100", distance 20, metric 0
  Tag 65002, type external
  Last update from 2.2.2.2 00:00:25 ago
  Routing Descriptor Blocks:
  * 172.64.2.1 (CUST2), from 172.64.2.1, 00:00:25 ago
      Route metric is 0, traffic share count is 1
      AS Hops 1
      Route tag 65002
    2.2.2.2 (Default-IP-Routing-Table), from 2.2.2.2, 00:00:25 ago
      Route metric is 0, traffic share count is 1
      AS Hops 1
      Route tag 65002
PE1# show ip route vrf CUST1
Routing Table: CUST1
---- output ommitted ----
Gateway of last resort is not set
     172.64.0.0/30 is subnetted, 3 subnets
C       172.64.1.0 is directly connected, Serial0/0
B       172.64.2.0 is directly connected, 00:22:35, Serial0/1
B       172.64.3.0 [200/0] via 2.2.2.2, 00:04:33
     10.0.0.0/24 is subnetted, 2 subnets
B       10.2.2.0 [20/0] via 172.64.2.1 (CUST2), 00:22:35
                 [20/0] via 2.2.2.2, 00:00:27
B       10.1.1.0 [20/0] via 172.64.1.1, 00:22:45

Notice both routes are installed in the routing table with BGP AD 20 although the second route is advertised by PE2 router through iBGP peering. The router does this to trick BGP into thinking that both routes are eBGP routes, otherwise, the route with AD 200 will never be installed in the routing table.

CEF fits into the bill:

Although PE1 router has multiple paths to 10.2.2.0/24 in the routing table, it still uses only one path to forward the traffic to that network. The reason being CEF uses per-destination load-sharing mechanism by default, i.e. traffic from a specific source-destination combination will always follow a specific path even if there are multiple paths available.

show ip cef

PE1# show ip cef vrf CUST1 10.2.2.0
10.2.2.0/24, version 21, epoch 0, per-destination sharing
0 packets, 0 bytes
  tag information set, one or more rewrites missing
    local tag: VPN-route-head
  via 172.64.2.1, 0 dependencies, recursive
    traffic share 1
    next hop 172.64.2.1, Serial0/1 via 172.64.2.0/30
    valid adjacency
    tag rewrite with Se0/1, point2point, tags imposed: {}
  via 2.2.2.2, 0 dependencies, recursive
    traffic share 1
    next hop 192.168.1.2, FastEthernet0/0 via 2.2.2.2/32
    valid adjacency
    tag rewrite with Fa0/0, 192.168.1.2, tags imposed: {20}
  0 packets, 0 bytes switched through the prefix
  tmstats: external 0 packets, 0 bytes
           internal 0 packets, 0 bytes

When a trace is run from CE1 router to 10.2.2.0, the PE1 router will forward all the packets along the same path every time.

per-destination load-sharing

CE1# traceroute 10.2.2.1
Type escape sequence to abort.
Tracing the route to 10.2.2.1
  1 172.64.1.2 220 msec 212 msec 456 msec
  2 172.64.2.1 [AS 100] 496 msec 448 msec *

When per-packet load-sharing is enabled on outgoing interfaces of PE1 router, all the packets it receives will be forwarded along different interfaces in case of load-sharing.

per-packet load-sharing on PE1

interface serial 0/1
 ip load-sharing per-packet
!
interface fastethernet 0/0
 ip load-sharing per-packet
!
PE1# clear ip cef 10.2.2.0 255.255.255.0 prefix-statistics
PE1# show ip cef vrf CUST1 10.2.2.0
10.2.2.0/24, version 73, epoch 0, per-packet sharing
0 packets, 0 bytes
  tag information set
    local tag: VPN-route-head
  via 172.64.2.1, 0 dependencies, recursive
    traffic share 1, current path
    next hop 172.64.2.1, Serial0/1 via 172.64.2.0/30
    valid adjacency
    tag rewrite with Se0/1, point2point, tags imposed: {}
  via 2.2.2.2, 0 dependencies, recursive
    traffic share 1
    next hop 192.168.1.2, FastEthernet0/0 via 2.2.2.2/32
    valid adjacency
    tag rewrite with Fa0/0, 192.168.1.2, tags imposed: {20}
  0 packets, 0 bytes switched through the prefix
  tmstats: external 0 packets, 0 bytes
           internal 0 packets, 0 bytes

Now when a trace is run from CE1 router to 10.2.2.0, packets are load-shared on per-packet basis at PE1.

traceroute output

CE1# traceroute 10.2.2.1
Type escape sequence to abort.
Tracing the route to 10.2.2.1
  1 172.64.1.2 60 msec 40 msec 48 msec
  2 172.64.2.1 [AS 100] 140 msec
    172.64.3.2 [AS 100] [MPLS: Label 20 Exp 0] 140 msec
    172.64.2.1 [AS 100] 116 msec