BGP Path Selection for Cloud Deployment

 Introduction

Requirement: Autonomous Systems AS1 and AS2 have two network paths with each other connected via two BGP routers in each AS. AS2 has two sets of IP prefixes, for example 10.10.10.0/24 (IP1) and 10.10.20.0/24 (IP2). How do we configure BGP so that traffic from AS1 towards IP1 destinations in AS2 exits over one path and towards IP2 destinations over the other path, thus load sharing the two connections.

This requirement can be satisfied with multiple different ways based on BGP protocol, attributes, routing  and route selection algorithm. For example, a) BGP attribute local-preference configurations in AS1, b) combination of community string (in AS2) and local-preference (in AS1) configurations, c) AS Path prepending configurations (in AS2). 

BGP network based on this particular requirement may be deployed in Cloud networks, where a customer may connect to a public Cloud provider (AWS, GCP, OCI, Azure, etc.) with dedicated (private) network connections (such as AWS Direct Connect or OCI FastConnect). BGP is then run over these connections.  In the above requirement there are two traffic paths, which, for example,  may be assigned to traffic for two different departments, dept1 and dept2 of an organization (AS1). The IP prefixes IP1 and IP2 may be assigned to resources that belong to dept1 and dept2 respectively and hosted in a public Cloud (AS2). The BGP routers in AS1 has to be configured in a way so that above path requirement is satisfied. We consider BGP configuration for selecting paths AS1-AS2-IP1 and AS1-AS2-IP2. We do not consider BGP configuration for reverse path selection. Note, this particular use case is relatively simpler than other general BGP network deployment use cases or topologies.

Also see BGP Network Design Assistant with GenAI-LLM

Topology

With BGP Local Preference

1. On R21 of AS2: 

   1. Match incoming prefix (CIDR) 10.10.10.0/24 advertised over eBGP from R11 and set Local Preference to, for example, 200.

   2. Match incoming prefix (CIDR) 10.10.20.0/24 advertised over eBGP from R11 and set Local Preference to, for example, 150.

2. On R22 of AS2: 

   1. Match incoming prefix (CIDR) 10.10.10.0/24 advertised over eBGP from R12 and set Local Preference to, for example, 150.

   2. Match incoming prefix (CIDR) 10.10.20.0/24 advertised over eBGP from R12 and set Local Preference to, for example, 200.

With this configuration route entries in R21 and R22 will be something like following.

Assume networks:

R21-R11: 10.0.15.0/24, R22-R12: 10.0.16.0/24, R21-R22: 10.0.14.0/24, R11-R12: 10.0.13.0/24

R21:

Network             Next Hop      Metric   LocPrf   AS_Path

10.10.10.0/24    10.0.15.2       0             200        65001 i

                             10.0.14.2      0             150        65001 i

10.10.20.0/24    10.0.14.2       0             200        65001 i

                     10.0.15.2       0             150        65001 i

R22:

Network            Next Hop       Metric   LocPrf   AS_Path

10.10.20.0/24   10.0.16.2        0             200        65001 i

                           10.0.14.1        0             150        65001  i

10.10.10.0/24   10.0.14.1        0            200         65001 i

                 10.0.16.2        0            150         65001 i

If traffic enters R21 towards 10.10.10.0/24, it will take R21-R11 path as path via R22 (advertised over iBGP) has lower LocPrf value (200<150).

Note, path selection to be effective LocPrf values have been flipped in R21 and R22 configurations.

See below for (Cisco) CLI template.

With BGP MED 

In this option MED metrics will be advertised by AS1 (not AS2). AS2 will then route based on MED metric (higher MED path is selected).

Route entries will look something like this:

R21:

Network             Next Hop      Metric   LocPrf   AS_Path

10.10.10.0/24    10.0.15.2       200                      65001 i

                             10.0.14.2      150                      65001 i

10.10.20.0/24    10.0.14.2       200                     65001 i

                     10.0.15.2       150                     65001 i

R22:

Network            Next Hop       Metric   LocPrf   AS_Path

10.10.20.0/24   10.0.16.2        200                       65001 i

                           10.0.14.1        150                       65001  i

10.10.10.0/24   10.0.14.1        200                       65001 i

                 10.0.16.2        150                       65001 i

See below for (Cisco) CLI template.

With Community String and Local Preference

In this option AS1 advertises towards AS2 a BGP community string for each prefix. AS2 then configures proper Local Preference based on relevant community string.

1. On R11 of AS1: 

   1. In outgoing direction set Community String for prefix  10.10.10.0/24 to, for example, 100:200 and advertise over eBGP to R21.

   2. In outgoing direction set Community String for prefix  10.10.20.0/24 to, for example, 100:150 and advertise over eBGP to R21.

2. On R12 of AS1: 

   1. In outgoing direction set Community String for prefix 10.10.10.0/24 to, for example, 100:150 and advertise over eBGP to R22.

   2. In outgoing direction set Community String for prefix 10.10.20.0/24 to, for example, 100:200 and advertise over eBGP to R22.

3. On R21 of AS2: 

   1. In incoming direction match Community String 100:200 from R11 and set Local Preference to, for example, 200. Hence in R21 we will have 10.10.10.0/24-->100:200-->200.

   2. In incoming direction match Community String 100:150 from R11 and set Local Preference to, for example, 150. Hence in R21 we will have 10.10.20.0/24-->100:150-->200.

4. On R22 of AS2: 

   1. In incoming direction match Community String 100:200 from R12 and set Local Preference to, for example, 200. Hence in R22 we will have 10.10.20.0/24-->100:200-->200.

   2. In incoming direction match Community String 100:150 from R11 and set Local Preference to, for example, 150. Hence in R22 we will have 10.10.10.0/24-->100:150-->150.

As a result R21 and R22 will have similar route entries as with Local Preference option described above.

With AS Path Prepending

1. On R11 of AS1: 

   1. In outgoing direction set AS Path for prefix  10.10.10.0/24 to, for example, 65001 and advertise over eBGP to R21.

   2. In outgoing direction set AS Path for prefix  10.10.20.0/24 to, for example, 65001 65001 (AS Path prepending) and advertise over eBGP to R21.

2. On R12 of AS1: 

   1. In outgoing direction set AS Path for prefix 10.10.10.0/24 to, for example, 65001 65001 and advertise over eBGP to R22.

   2. In outgoing direction set AS Path for prefix 10.10.20.0/24 to, for example, 65001 and advertise over eBGP to R22.

R21:

Network             Next Hop      Metric   LocPrf   AS_Path

10.10.10.0/24    10.0.15.2                                   65001 i

                             10.0.14.2                                  65001 65001 i

10.10.20.0/24    10.0.14.2                                  65001 i

                     10.0.15.2                                   65001 65001 i

R22:

Network            Next Hop       Metric   LocPrf   AS_Path

10.10.20.0/24   10.0.16.2                                     65001 i

                           10.0.14.1                                     65001  65001 i

10.10.10.0/24   10.0.14.1                                     65001 i

                 10.0.16.2                                     65001 65001 i

See below for (Cisco) CLI template.

Configuration Template with CLI

CLI template: With Local Preference

R21:

router bgp 65002

 neighbor 10.0.14.2 remote-as 65001 ! iBGP

 neighbor 10.0.14.2 next-hop-self

 neighbor 10.0.15.2 remote-as 65001

 neighbor 10.0.15.2 route-map setLocalPref-R21 in

access-list 1 permit 10.10.10.0/24 0.255.255.255

access-list 2 permit 10.10.20.0/24 0.255.255.255

route-map setLocalPref-R21 permit 10

 match ip address 1

 set local-preference 200

!

route-map setLocalPref-R21 permit 20

 match ip address 2

 set local-preference 150

R22:

router bgp 65002

 neighbor 10.0.14.1 remote-as 65001

 neighbor 10.0.14.1 next-hop-self

 neighbor 10.0.16.2 remote-as 65001

 neighbor 10.0.16.2 route-map setLocalPref-R22 in

access-list 1 permit 10.10.10.0/24 0.255.255.255

access-list 2 permit 10.10.20.0/24 0.255.255.255

route-map setLocalPref-R22 permit 10

 match ip address 1

 set local-preference 150

route-map setLocalPref-R22 permit 20

 match ip address 2

 set local-preference 200

With MED

R11:

router bgp 65001

 neighbor 10.0.13.2 remote-as 65001 ! iBGP

 neighbor 10.0.13.2 next-hop-self

 neighbor 10.0.15.2 remote-as 65002

 neighbor 10.0.15.2 route-map setMED-R11 out

access-list 1 permit 10.10.10.0/24 0.255.255.255

access-list 2 permit 10.10.20.0/24 0.255.255.255

route-map setMED-R11 permit 10

 match ip address 1

 set metric 200

route-map setMED-R11 permit 20

 match ip address 2

 set metric 150

R12:

router bgp 65001

 neighbor 10.0.13.1 remote-as 65001 ! iBGP

 neighbor 10.0.13.1 next-hop-self

 neighbor 10.0.16.2 remote-as 65002

 neighbor 10.0.16.2 route-map setMED-R12 out

access-list 1 permit 10.10.10.0/24 0.255.255.255

access-list 2 permit 10.10.20.0/24 0.255.255.255

route-map setMED-R12 permit 10

 match ip address 1

 set local-preference 150

route-map setMED-R12 permit 20

 match ip address 2

 set local-preference 200

With AS Path Prepending

R11:

router bgp 65001

 neighbor 10.0.13.2 remote-as 65001 ! iBGP

 neighbor 10.0.13.2 next-hop-self

 neighbor 10.0.15.2 remote-as 65002

 neighbor 10.0.15.2 route-map setASPrepend-R21 out

access-list 1 permit 10.10.10.0/24 0.255.255.255

access-list 2 permit 10.10.20.0/24 0.255.255.255

route-map setASPrepend-R21 permit 10

 match ip address 1

 set as-path prepend 65001

r route-map setASPrepend-R21 permit 20

 match ip address 2

 set as-path prepend 65001 65001

R12:

router bgp 65001

 neighbor 10.0.13.1 remote-as 65001 ! iBGP

 neighbor 10.0.13.1 next-hop-self

 neighbor 10.0.16.2 remote-as 65002

 neighbor 10.0.16.2 route-map setASPrepend-R12 out

access-list 1 permit 10.10.10.0/24 0.255.255.255

access-list 2 permit 10.10.20.0/24 0.255.255.255

route-map setASPrepend-R12 permit 10

 match ip address 1

 set as-path prepend 65001

route-map setASPrepend-R12 permit 20

 match ip address 2

 set as-path prepend 65001 65001

With Community String and Local Preference

R11:

router bgp 65001

 neighbor 10.0.13.2 remote-as 65001 ! iBGP

 neighbor 10.0.13.2 next-hop-self

 neighbor 10.0.15.2 remote-as 65002

 neighbor 10.0.15.2 send-community

 neighbor 10.0.15.2 route-map setCommunity-R11 out

access-list 1 permit 10.10.10.0/24 0.255.255.255

access-list 2 permit 10.10.20.0/24 0.255.255.255

route-map setCommunity-R11 permit 10

 match ip address 1

 set community 100:200

route-map setCommunity-R11 permit 10

 match ip address 2

 set community 100:150

R12:

router bgp 65001

neighbor 10.0.13.1 remote-as 65001 ! iBGP

 neighbor 10.0.13.1 next-hop-self

 neighbor 10.0.16.2 remote-as 65002

 neighbor 10.0.16.2 send-community

 neighbor 10.0.16.2 route-map setCommunity-R12 out

access-list 1 permit 10.10.10.0/24 0.255.255.255

access-list 2 permit 10.10.20.0/24 0.255.255.255

route-map setCommunity-R12 permit 10

 match ip address 1

 set community 100:150

route-map setCommunity-R12 permit 20

 match ip address 2

 set community 100:200

With Community String and Local Preference

R21:

router bgp 65002

 neighbor 10.0.14.2 remote-as 65001 ! iBGP

 neighbor 10.0.14.2 next-hop-self

 neighbor 10.0.15.2 remote-as 65001

 neighbor 10.0.15.2 route-map setLocalPref-R21 in

ip community-list 1 permit 100:200

ip community-list 2 permit 100:150

route-map setLocalPref-R21 permit 10

 match community 1

 set local-preference 200

route-map setLocalPref-R21 permit 20

 match community 2

 set local-preference 150

R22:

router bgp 65002

 neighbor 10.0.14.1 remote-as 65001

 neighbor 10.0.14.1 next-hop-self

 neighbor 10.0.16.2 remote-as 65001

 neighbor 10.0.16.2 route-map setLocalPref-R22 in

ip community-list 1 permit 100:200

ip community-list 2 permit 100:150

route-map setLocalPref-R22 permit 10

 match community 1

 set local-preference 150

route-map setLocalPref-R22 permit 20

 match community 2

 set local-preference 200

BGP Path Selection Algorithm

BGP best route or path selection algorithm: 

Following is the order of BGP best route selection algorithm (not all steps are shown). Once a step is satisfied, the algorithm stops evaluating further. BGP speakers in local AS (iBGP) and different ASs (eBGP) exchange BGP routes between each other. A speaker may receive routes from multiple other speakers for the same prefix. Given a destination prefix or CIDR, there may be more than one route entry to the destination. Using the algorithm described below a BGP speaker decides which route to choose to advertise to its peers or neighbors. This same route is installed in the forwarding table to route IP traffic to neighbors.

1. Weight configured on a speaker: route entry with the highest weight is selected. This weight is not advertised to the iBGP neighbors, that is, its scope is only the speaker where it is configured. It is also applicable to Cisco routers only.

2. Local Preference number or LOCAL_PREF:  route entry with the highest LOCAL_PREF number is selected. Its scope is local AS only. A BGP speaker advertises the LOCAL_PREF to its iBGP neighbors. It is used to indicate which router to use to exit the AS (see MED below in comparison).

3. Route origin: route entry that was locally originated via a network or aggregate BGP subcommand or through redistribution from an IGP is selected. Local paths that are sourced by the network or redistribute commands are preferred over local aggregates that are sourced by the aggregate-address command.

4. AS Path:  AS Path consists of a list of ASN listing the AS numbers of the ASs through which the route traversed or originated. A route entry with the shortest AS Path length (list of ASN with the minimum length) is selected. ASNs in the AS Path can be overridden by the local AS. More ASNs, typically local ASN can be added to an AS path to influence path selection by upstream routers. AS Path can be ignored by configuration.

5. Origin type: route entry with the lowest origin type is selected. IGP is lower than EGP, which is lower than INCOMPLETE. Prefer the route with the lower origin code. Routes learned from an IGP have a lower origin code than those learned from an exterior gateway protocol (EGP), and both have lower origin codes than incomplete routes (those whose origin is unknown).

6. Multi-exit discriminator (MED): route entry with the lowest MED metric will be selected. MED is used to influence traffic ingress into an AS (whereas LOCAL_PREF, which is local to an AS only is used to select an exit router to exit to another AS). MED is a 4 byte field. MED is not advertised transitively to other eBGP peers, but advertised to iBGP peers. Unequal MEDs advertised across, for example, two sessions will typically result in primary and backup paths. If load-balancing (LB) is desired and routers support LB, MED should be set to equal values or zero and ASN Path of same length. MED attribute is applied to the outbound interface and shows the best inbound interface into its AS. Metric attribute (of show ip bgp) shows MED value. … When all other factors are equal, the exit point with the lowest MED is preferred

7. eBGP vs iBGP: route entry learned over eBGP is preferred over that of iBGP.

8. Prefer the path with the lowest IGP metric to the BGP next hop.

9. Determine if multiple paths require installation in the routing table for BGP Multipath.

10. When both paths are external, prefer the path that was received first (the oldest one).

11. Prefer the route that comes from the BGP router with the lowest router ID.

12. If the originator or router ID is the same for multiple paths, prefer the path with the minimum cluster list length.
    This is only present in BGP RR environments. It allows clients to peer with RRs or clients in other clusters. In this scenario, the client must be aware of the RR-specific BGP attribute.

13. Prefer the path that comes from the lowest neighbor address.