BGP Multihoming and Multipath

BGP Multipath

Unlike most routing protocols, BGP only selects a single best path for each prefix. It doesn’t do ECMP (Equal Cost Multi-Path Routing) by default but it is possible to enable this.

The simplest method of providing redundancy is to provide a second circuit. Adding a second circuit and establishing a second BGP session across that peering link is known as BGP multihoming because there are multiple sessions to learn routes and establish connectivity.

BGP’s default behavior is to advertise only the best path to the routing information base (RIB), which means that only one path for a network prefix is used when forwarding network traffic to a destination.

BGP multipath allows for multiple paths to be presented to the RIB, so that both paths can forward traffic to a network prefix at the same time. BGP multipath is an enhanced form of BGP multihoming.

Note: It is vital to understand that the primary difference between BGP multihoming and BGP multipath is how load balancing works. BGP multipath attempts to distribute the load of the traffic dynamically. BGP multihoming is distributed somewhat by the nature of the BGP best path algorithm, but manipulation to the inbound/outbound routing policies is required to reach a more equally distributed load among the links.

There are three types of BGP multipath:

  1. External BGP (EBGP)
  2. Internal BGP (IBGP)
  3. EIBGP multipath

EBGP multipath and IBGP multipath cannot be configured on the router at the same time.

In order for BGP to use the second path, the following attributes have to match:

  • Weight
  • Local Preference
  • AS Path (both AS numbers and path length)
  • Origin Code
  • MED
  • IGP Metric

Illustrates the differences between BGP multihoming and multipath from four common scenarios, as follows:

  • Scenario 1: R1 connects to two different Internet SPs with two different circuits. R1 cannot use EBGP multipath because the AS-Paths are different for any prefix on the Internet. R1 is multihomed only.

  • Scenario 2: R1 connects to the same Internet SP with two different circuits. R1 can use EBGP multipath for any Internet prefixes because the routes are learned via the same service provider (regardless whether R1 connects to the same router or different routers in service provider ABC). This assumes that all other BGP PAs are identical between R1’s perspective.

  • Scenario 3: R3 connects via IBGP to R1 and R2. R1 connects to SP ABC, whereas R2 connects to SP XYZ. R3 cannot use IBGP multipath because the AS-Paths are different for any Internet prefixes. R3 is multihomed only.

  • Scenario 4: R3 connects via IBGP to R1 and R2. R1 and R2 connect to the same Internet SP. R3 uses IBGP multipath for any Internet prefixes because the AS-Path is the same (along with the other required BGP PAs.)
BGP Multipath and BGP Multihome

BGP Multipath

  • R1 receives two paths from AS20 (via R4 and R5)
  • Best-path algorithm selects one and installs it in routing table
    • By default, all of the traffic goes via one link only
    • Assuming all attributes are equal, uses the one from the lower neighbor IP address

EBGP Multipath

  • Enable EBGP multipath on R1 to install both paths
    • router bgp 10
      • maximumpaths 2
  • Multipath selection is part of the Best Path algorithm
    • Evaluated before the more arbitrary tie breakers like IP address/etc.
  • Only paths with identical ASPATH will be considered
    • Hidden knob “bgp bestpath as-path multipath relax” changes this, but be aware of what you’re doing

IBGP Multipath

  • In this topology, EBGP Multipath will not help
  • R1 will choose one of the internal paths, and will select one: R2 or R3
  • If R1’s IGP cost to R2 and R3 is equal, and all other path attributes are the same, IBGP multipath can be used
    • router bgp 10
      • maximum-paths ibgp 2

EBGP and IBGP Multipath Configuration

EBGP Same AS

				
					R1#show bgp ipv4 unicast summary 
/* BGP router identifier 10.2.2.1, local AS number 1
BGP table version is 2, main routing table version 2
1 network entries using 144 bytes of memory
2 path entries using 168 bytes of memory
1/1 BGP path/bestpath attribute entries using 160 bytes of memory
1 BGP AS-PATH entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 496 total bytes of memory
BGP activity 1/0 prefixes, 2/0 paths, scan interval 60 secs */

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.1.1.2        4          100      14      14        2    0    0 00:08:42        1
10.2.2.2        4          100      14      12        2    0    0 00:08:20        1
R1#

R1#show bgp ipv4 unicast 
/* BGP table version is 2, local router ID is 10.2.2.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
              t secondary path, 
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found */

     Network          Next Hop            Metric LocPrf Weight Path
 *    192.168.23.0     10.2.2.2                 0             0 100 i
 *>                    10.1.1.2                 0             0 100 i
R1#

R1#show bgp ipv4 unicast 192.168.23.0/24
BGP routing table entry for 192.168.23.0/24, version 2
Paths: (2 available, best #2, table default)
  Advertised to update-groups:
     1         
  Refresh Epoch 1
  100
    10.2.2.2 from 10.2.2.2 (192.168.23.3)
      Origin IGP, metric 0, localpref 100, valid, external
      rx pathid: 0, tx pathid: 0
  Refresh Epoch 1
  100
    10.1.1.2 from 10.1.1.2 (192.168.23.2)
      Origin IGP, metric 0, localpref 100, valid, external, best
      rx pathid: 0, tx pathid: 0x0
R1#

// EBGP Multipath Configured

R1#show bgp ipv4 unicast                
/* BGP table version is 3, local router ID is 10.2.2.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
              t secondary path, 
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found */

     Network          Next Hop            Metric LocPrf Weight Path
 *m   192.168.23.0     10.2.2.2                 0             0 100 i
 *>                    10.1.1.2                 0             0 100 i
R1#

R1#show ip route bgp
/* output omitted */
Gateway of last resort is not set

B     192.168.23.0/24 [20/0] via 10.2.2.2, 00:01:34
                      [20/0] via 10.1.1.2, 00:01:34
R1#
				
			

EBGP Different AS

				
					R1#show bgp ipv4 unicast summary 
/* BGP router identifier 10.2.2.1, local AS number 1
BGP table version is 2, main routing table version 2
1 network entries using 144 bytes of memory
2 path entries using 168 bytes of memory
2/1 BGP path/bestpath attribute entries using 320 bytes of memory
2 BGP AS-PATH entries using 48 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 680 total bytes of memory
BGP activity 1/0 prefixes, 2/0 paths, scan interval 60 secs */

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.1.1.2        4            2       5       6        2    0    0 00:02:35        1
10.2.2.2        4            3       9       8        2    0    0 00:04:32        1
R1#

R1#show bgp ipv4 unicast 
/* BGP table version is 2, local router ID is 10.2.2.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
              t secondary path, 
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found */

     Network          Next Hop            Metric LocPrf Weight Path
 *    192.168.23.0     10.2.2.2                               0 3 4 i
 *>                    10.1.1.2                               0 2 4 i
R1#

R1#show bgp ipv4 unicast 192.168.23.0/24
BGP routing table entry for 192.168.23.0/24, version 2
Paths: (2 available, best #2, table default)
  Advertised to update-groups:
     1         
  Refresh Epoch 1
  3 4
    10.2.2.2 from 10.2.2.2 (10.4.4.2)
      Origin IGP, localpref 100, valid, external
      rx pathid: 0, tx pathid: 0
  Refresh Epoch 1
  2 4
    10.1.1.2 from 10.1.1.2 (10.3.3.2)
      Origin IGP, localpref 100, valid, external, best
      rx pathid: 0, tx pathid: 0x0
R1#

R1#show ip route bgp
/* output omitted */
Gateway of last resort is not set
B     192.168.23.0/24 [20/0] via 10.1.1.2, 00:02:34
R1#

// Configured multipath but this command doesn't help

R1#show bgp ipv4 unicast 
/* BGP table version is 2, local router ID is 10.2.2.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
              t secondary path, 
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found */

     Network          Next Hop            Metric LocPrf Weight Path
 *    192.168.23.0     10.2.2.2                               0 3 4 i
 *>                    10.1.1.2                               0 2 4 i
R1#

R1#show ip route bgp
/* output omitted */
Gateway of last resort is not set
B     192.168.23.0/24 [20/0] via 10.1.1.2, 00:00:04
R1#

/*
The problem here is that we have two different AS numbers, AS 2 and AS 3.
We can tell BGP to “relax” its requirement of having the same AS path numbers
and AS path length to only checking the AS path length.
This can be done with the following hidden command:
bgp bestpath as-path multipath-relax
*/

R1#show bgp ipv4 unicast     
/* BGP table version is 3, local router ID is 10.2.2.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
              t secondary path, 
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found */

     Network          Next Hop            Metric LocPrf Weight Path
 *m   192.168.23.0     10.2.2.2                               0 3 4 i
 *>                    10.1.1.2                               0 2 4 i
R1#

R1#show ip route bgp
/* output omitted */
Gateway of last resort is not set
B     192.168.23.0/24 [20/0] via 10.2.2.2, 00:00:44
                      [20/0] via 10.1.1.2, 00:00:44
R1#
				
			

AS-Path Relax

BGP multipath works well if an organization uses the same service provider for Internet connectivity. However, if an organization uses different service providers, the AS-Path is not identical and does not meet the requirements for BGP multipath. The AS-Path relax feature allows for BGP multipath to work when the AS-Path length is the same, but the AS-Paths are different.

The AS-Path relax feature is configured with the BGP configuration command bgp bestpath as-path multipath-relax on IOS, IOS XR, and NX-OS devices.

IBGP

				
					R1#show bgp ipv4 unicast summary 
/* BGP router identifier 10.2.2.1, local AS number 1
BGP table version is 2, main routing table version 2
1 network entries using 144 bytes of memory
2 path entries using 168 bytes of memory
1/1 BGP path/bestpath attribute entries using 160 bytes of memory
1 BGP AS-PATH entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 496 total bytes of memory
BGP activity 1/0 prefixes, 2/0 paths, scan interval 60 secs */

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.1.1.2        4            1       9       9        2    0    0 00:05:50        1
10.2.2.2        4            1       8       9        2    0    0 00:04:50        1
R1#

R1#show bgp ipv4 unicast 
/* BGP table version is 2, local router ID is 10.2.2.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
              t secondary path, 
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found */

     Network          Next Hop            Metric LocPrf Weight Path
 * i  192.168.23.0     10.2.2.2                 0    100      0 2 i
 *>i                   10.1.1.2                 0    100      0 2 i
R1#

R1#show bgp ipv4 unicast 192.168.23.0/24
BGP routing table entry for 192.168.23.0/24, version 2
Paths: (2 available, best #2, table default)
  Not advertised to any peer
  Refresh Epoch 1
  2
    10.2.2.2 from 10.2.2.2 (10.4.4.2)
      Origin IGP, metric 0, localpref 100, valid, internal
      rx pathid: 0, tx pathid: 0
  Refresh Epoch 1
  2
    10.1.1.2 from 10.1.1.2 (10.3.3.2)
      Origin IGP, metric 0, localpref 100, valid, internal, best
      rx pathid: 0, tx pathid: 0x0
R1#

R1#show ip route bgp
/* output omitted */
Gateway of last resort is not set
B     192.168.23.0/24 [200/0] via 10.1.1.2, 00:01:58
R1#

// IBGP Multipath configured

R1#show bgp ipv4 unicast 
/* BGP table version is 3, local router ID is 10.2.2.1
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal, 
              r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter, 
              x best-external, a additional-path, c RIB-compressed, 
              t secondary path, 
Origin codes: i - IGP, e - EGP, ? - incomplete
RPKI validation codes: V valid, I invalid, N Not found */

     Network          Next Hop            Metric LocPrf Weight Path
 *mi  192.168.23.0     10.2.2.2                 0    100      0 2 i
 *>i                   10.1.1.2                 0    100      0 2 i
R1#

R1#show ip route bgp 
/* output omitted */
Gateway of last resort is not set
B     192.168.23.0/24 [200/0] via 10.2.2.2, 00:00:23
                      [200/0] via 10.1.1.2, 00:00:23
R1#
				
			

EIBGP Multipath

The rules of IBGP or EBGP multipath prevents a router from installing an EBGP and IBGP route into the RIB at the same time. However, in some environments this can prevent an organization from using the full bandwidth available. EIBGP multipath allows for an EBGP and IBGP path with unequal next-hop metrics to be selected as the best path.

EIBGP multipath configuration on IOS and IOS XR routers uses the BGP address-family configuration command maximum-paths eibgp number-paths.

Figure used to explain how EIBGP works further and provides the following information:

  • R1, R2, R3, R4, and R5 have established an IBGP full mesh for AS100.
  • R1, R3, and R5 set the next-hop-self to IBGP peers.
  • AS100 routers are using OSPF as the IGP.
  • EIBGP maximum paths have been set to 5 for all routers in AS100.
  • The BGP table for all the routers is displayed after BGP best path calculation.

R1 and R2 installed only one BGP path for 172.16.0.0/24 network, whereas R3, R4, and R5 utilized EIBGP multipath and installed multiple routes.

R1

  • R1 identifies the EBGP path via 10.100.1.1 as the best path because EBGP paths are preferred over IBGP paths.
  • Notice that the other 172.16.0.0/24 paths are ineligible for multipath installation because the AS-Path does not match the AS-Path of the best path (300 400).

R2

  • R2 identifies the IBGP path via R1 as the best path because the IGP metric to R1 is lower than the IGP metric to R3 and R5.
  • Notice that the other 172.16.0.0/24 paths are ineligible for multipath because the AS-Path does not match the AS-Path of the best path (300 400).

R3

  • R3 identifies the BGP best path via 10.100.3.1 as the best path because EBGP paths are preferred over IBGP paths.
  • A second 172.16.0.0/24 path has an identical AS-Path to the BGP best path (300 400) and can be installed into the RIB even though it uses an IBGP path.

Note: If an IBGP route is used with a primary EBGP path, the administrative distance (AD) is changed to 20 so that both routes install into the RIB.

Note: IOS XR routers do not display the m flag (status code) for EIBGP multipath routes in the BGP output.

R4

  • R4 identifies the BGP best path through R3 because the IGP cost is lower than the path through R5.
  • A second 172.16.0.0/24 path has an identical AS-Path to the BGP best path (300 400) and can be installed into the RIB even though the IGP metric is different from the metric to R3.

R5

  • R5 identifies the BGP best path via 10.100.5.1 as the best path because the EBGP path is preferred over IBGP paths.
  • R3’s 172.16.0.0/24 path has an identical AS-Path to the BGP best path (300 400) and can be installed into the RIB even though it uses an IBGP path.

Note: The use of EIBGP multipath can introduce a routing loop into a topology and should be used only after mapping out all the traffic flows. EIBGP multipath is typically used in the context of MPLS L3VPNs. Using this feature from the global table could result in routing loops if not designed properly.

MPLS L3VPN topology to demonstrate an EIBGP multipath real-world use case

  • All the customer edge (CE) routers belong to the same VRF instance that is hosted on the provider edge (PE) routers.
  • Network traffic from CE3, CE4, and CE5 to 172.16.1.0/24 could saturate the network link between PE2 and CE1.
  • PE2 selects this path because of the EBGP peering versus the paths by PE1 and PE3 that are IBGP paths.
  • By enabling EIBGP multipath on PE2, traffic can be distributed across all three links. The customer prefers this because it does not have to increase bandwidth on the circuits between CE1 and PE2.
EIBGP Multipath with MPLS L3VPN Topology
Tags: ,

Leave a Reply

Related Post

Policy ControlPolicy Control

Policy Control Using BGP Attributes Most commonly used BGP attributes: LOCAL_PREF AS_PATH MED ORIGIN NEXT_HOP WEIGHT BGP picks a best path for a destination IP prefix from multiple paths and

BGP CommunitiesBGP Communities

BGP Communities BGP communities are an optional transitive BGP attribute that can traverse from autonomous system to autonomous system. A BGP community is a 32-bit number that can be included