BGP Multipath Load Sharing for Both eBGP and iBGP in an MPLS-VPN
BGP Multipath Load Sharing for Both eBGP and iBGP in an MPLS-VPN
A BGP routing process installs only a single path as the best path in the routing table by default. Also, BGP prefers eBGP paths over iBGP paths by default. The maximum-paths command allows BGP to install multiple paths in the routing table, however, it will still advertise only ONE BEST path to its eBGP neighbor. The multipath load-sharing for both eBGP and iBGP allows to configure multipath load-balancing with both external and internal paths.
Network topology:
CE1 and CE2 routers are two customer devices of different organizations, which are connected to the SP, offering an MPLS VPN service. CE2 is multihomed that is connected to two SP routers. There is an extranet-VPN connection between CE1 and CE2 routers.
Initial Configuration:
CE1 & CE2 configuration
CE1 router:
interface serial 0/0
ip address 172.64.1.1 255.255.255.252
!
router bgp 65001
neighbor 172.64.1.2 remote-as 100
network 10.1.1.0 mask 255.255.255.0
!
CE2 router:
interface serial 0/0
ip address 172.64.2.1 255.255.255.252
!
interface serial 0/1
ip address 172.64.3.1 255.255.255.252
!
router bgp 65002
neighbor 172.64.2.2 remote-as 100
neighbor 172.64.3.2 remote-as 100
network 10.2.2.0 mask 255.255.255.0
!
PE1 configuration
ip vrf CUST1
rd 1:1
route-target both 1:1
route-target import 1:2
route-target import 2:2
!
ip vrf CUST2
rd 1:2
route-target both 1:2
route-target import 1:1
!
interface Loopback 0
ip address 1.1.1.1 255.255.255.255
!
interface serial 0/0
ip vrf forwarding CUST1
ip address 172.64.1.2 255.255.255.252
!
interface serial 0/1
ip vrf forwarding CUST2
ip address 172.64.2.2 255.255.255.252
!
router bgp 100
neighbor 2.2.2.2 remote-as 100
neighbor 2.2.2.2 update-source Loopback 0
!
address-family vpnv4
neighbor 2.2.2.2 activate
neighbor 2.2.2.2 send-community both
exit-address-family
!
address-family ipv4 vrf CUST1
neighbor 172.64.1.1 remote-as 65001
neighbor 172.64.1.1 activate
redistribute connected
exit-address-family
!
address-family ipv4 vrf CUST2
neighbor 172.64.2.1 remote-as 65002
neighbor 172.64.2.1 activate
redistribute connected
exit-address-family
!
PE2 configuration
ip vrf CUST2
rd 2:2
route-target both 2:2
route-target import 1:1
!
interface Loopback 0
ip address 2.2.2.2 255.255.255.255
!
interface serial 0/0
ip vrf forwarding CUST2
ip address 172.64.3.2 255.255.255.252
!
router bgp 100
neighbor 1.1.1.1 remote-as 100
neighbor 1.1.1.1 update-source Loopback 0
!
address-family vpnv4
neighbor 1.1.1.1 activate
neighbor 1.1.1.1 send-community both
exit-address-family
!
address-family ipv4 vrf CUST2
neighbor 172.64.3.1 remote-as 65002
neighbor 172.64.3.1 activate
redistribute connected
exit-address-family
!
Default Behavior:
PE1 router receives 10.2.2.0/24 network from CE2 router over its eBGP connection imported from vrf CUST2. It installs this route as VPNv4 route in its VRF CUST1 BGP table as 1:2:10.2.2.0/24 with next-hop as CE2 router (172.64.2.1).
CE2 also advertises 10.2.2.0/24 to PE2 router over eBGP connection. PE2 router also installs this route as VPNv4 route in its VRF CUST1 BGP table as 2:2:10.2.2.0/24 with next-hop as PE2 router (2.2.2.2). Since there is an iBGP peering between PE1 and PE2 routers, PE2 router also advertises this VPNv4 route to PE1 router. PE1 router is configured to accept routes with RD 2:2. Hence, PE1 router will install this VPNv4 route in its BGP table.
However, by default, BGP will install only one route in the VRF routing table of CUST1.
10.2.2.0/24 on PE1
PE1# show ip bgp vpnv4 vrf CUST1 10.2.2.0
BGP routing table entry for 1:1:10.2.2.0/24, version 13
Paths: (2 available, best #1, table CUST1)
Advertised to update-groups:
2
65002, imported path from 1:2:10.2.2.0/24
172.64.2.1 from 172.64.2.1 (10.2.2.1)
Origin IGP, metric 0, localpref 100, valid, external, best
Extended Community: RT:1:2
65002, imported path from 2:2:10.2.2.0/24
2.2.2.2 (metric 11) from 2.2.2.2 (2.2.2.2)
Origin IGP, metric 0, localpref 100, valid, internal
Extended Community: RT:2:2
mpls labels in/out nolabel/20
PE1# show ip route vrf CUST1
Routing Table: CUST1
---- output omitted ----
Gateway of last resort is not set
172.64.0.0/30 is subnetted, 3 subnets
C 172.64.1.0 is directly connected, Serial0/0
B 172.64.2.0 is directly connected, 00:13:31, Serial0/1
B 172.64.3.0 [200/0] via 2.2.2.2, 00:06:45
10.0.0.0/24 is subnetted, 2 subnets
B 10.2.2.0 [20/0] via 172.64.2.1 (CUST2), 00:13:31
B 10.1.1.0 [20/0] via 172.64.1.1, 00:13:33
PE1 router will advertise only one path to 10.2.2.0/24 to CE1 router.
Enabling Multipath feature:
The multipath feature can be enabled using maximum-paths command under address-family for a particular VRF in BGP. It allows to store multiple paths for a particular prefix. The maximum-paths eibgp <value> command allows multiple eBGP and iBGP paths in the BGP table for a prefix.
Enabling multipath feature on PE1 for vrf CUST1
router bgp 100
!
address-family ipv4 vrf CUST1
maximum-paths eibgp 2
exit-address-family
!
After the multipath feature is enabled, PE1 router will install both above mentioned routes in its routing table.
10.2.2.0/24 on PE1
PE1# show ip route vrf CUST1 10.2.2.0
Routing entry for 10.2.2.0/24
Known via "bgp 100", distance 20, metric 0
Tag 65002, type external
Last update from 2.2.2.2 00:00:25 ago
Routing Descriptor Blocks:
* 172.64.2.1 (CUST2), from 172.64.2.1, 00:00:25 ago
Route metric is 0, traffic share count is 1
AS Hops 1
Route tag 65002
2.2.2.2 (Default-IP-Routing-Table), from 2.2.2.2, 00:00:25 ago
Route metric is 0, traffic share count is 1
AS Hops 1
Route tag 65002
PE1# show ip route vrf CUST1
Routing Table: CUST1
---- output ommitted ----
Gateway of last resort is not set
172.64.0.0/30 is subnetted, 3 subnets
C 172.64.1.0 is directly connected, Serial0/0
B 172.64.2.0 is directly connected, 00:22:35, Serial0/1
B 172.64.3.0 [200/0] via 2.2.2.2, 00:04:33
10.0.0.0/24 is subnetted, 2 subnets
B 10.2.2.0 [20/0] via 172.64.2.1 (CUST2), 00:22:35
[20/0] via 2.2.2.2, 00:00:27
B 10.1.1.0 [20/0] via 172.64.1.1, 00:22:45
Notice both routes are installed in the routing table with BGP AD 20 although the second route is advertised by PE2 router through iBGP peering. The router does this to trick BGP into thinking that both routes are eBGP routes, otherwise, the route with AD 200 will never be installed in the routing table.
CEF fits into the bill:
Although PE1 router has multiple paths to 10.2.2.0/24 in the routing table, it still uses only one path to forward the traffic to that network. The reason being CEF uses per-destination load-sharing mechanism by default, i.e. traffic from a specific source-destination combination will always follow a specific path even if there are multiple paths available.
show ip cef
PE1# show ip cef vrf CUST1 10.2.2.0
10.2.2.0/24, version 21, epoch 0, per-destination sharing
0 packets, 0 bytes
tag information set, one or more rewrites missing
local tag: VPN-route-head
via 172.64.2.1, 0 dependencies, recursive
traffic share 1
next hop 172.64.2.1, Serial0/1 via 172.64.2.0/30
valid adjacency
tag rewrite with Se0/1, point2point, tags imposed: {}
via 2.2.2.2, 0 dependencies, recursive
traffic share 1
next hop 192.168.1.2, FastEthernet0/0 via 2.2.2.2/32
valid adjacency
tag rewrite with Fa0/0, 192.168.1.2, tags imposed: {20}
0 packets, 0 bytes switched through the prefix
tmstats: external 0 packets, 0 bytes
internal 0 packets, 0 bytes
When a trace is run from CE1 router to 10.2.2.0, the PE1 router will forward all the packets along the same path every time.
per-destination load-sharing
CE1# traceroute 10.2.2.1
Type escape sequence to abort.
Tracing the route to 10.2.2.1
1 172.64.1.2 220 msec 212 msec 456 msec
2 172.64.2.1 [AS 100] 496 msec 448 msec *
When per-packet load-sharing is enabled on outgoing interfaces of PE1 router, all the packets it receives will be forwarded along different interfaces in case of load-sharing.
per-packet load-sharing on PE1
interface serial 0/1
ip load-sharing per-packet
!
interface fastethernet 0/0
ip load-sharing per-packet
!
PE1# clear ip cef 10.2.2.0 255.255.255.0 prefix-statistics
PE1# show ip cef vrf CUST1 10.2.2.0
10.2.2.0/24, version 73, epoch 0, per-packet sharing
0 packets, 0 bytes
tag information set
local tag: VPN-route-head
via 172.64.2.1, 0 dependencies, recursive
traffic share 1, current path
next hop 172.64.2.1, Serial0/1 via 172.64.2.0/30
valid adjacency
tag rewrite with Se0/1, point2point, tags imposed: {}
via 2.2.2.2, 0 dependencies, recursive
traffic share 1
next hop 192.168.1.2, FastEthernet0/0 via 2.2.2.2/32
valid adjacency
tag rewrite with Fa0/0, 192.168.1.2, tags imposed: {20}
0 packets, 0 bytes switched through the prefix
tmstats: external 0 packets, 0 bytes
internal 0 packets, 0 bytes
Now when a trace is run from CE1 router to 10.2.2.0, packets are load-shared on per-packet basis at PE1.
traceroute output
CE1# traceroute 10.2.2.1
Type escape sequence to abort.
Tracing the route to 10.2.2.1
1 172.64.1.2 60 msec 40 msec 48 msec
2 172.64.2.1 [AS 100] 140 msec
172.64.3.2 [AS 100] [MPLS: Label 20 Exp 0] 140 msec
172.64.2.1 [AS 100] 116 msec