Tuning BGP Memory

Scaling BGP Functions

BGP is a heavy protocol because it uses the most CPU and memory resources on a router. And there are many factors that explain why it keeps utilizing more and more resources. The three major factors for BGP memory consumption are as follows:

  1. Prefixes
  2. Paths
  3. Attributes

BGP can hold many prefixes, and each prefix consumes some amount of memory. But when the same prefix is learned via multiple paths, that information is also maintained in the BGP table. Each path requires additional memory space. Because BGP was designed to give control to each AS to manage the flow of traffic through various attributes, each prefix can have several attributes per path.

Example:

  • Example demonstrates various BGP paths with various communities and attributes.
  • Notice that each attribute or community learned from an individual neighbor for the prefix 192.168.100.0/24 is stored in the BGP table, thereby increasing memory consumption.
  • Output showed how the memory consumption per BGP prefix depends on the number of paths, path attributes, and AS-Paths associated with it. When the prefix scale increases, the memory consumption on the router increases, and services could be impacted. It is therefore imperative to have the BGP memory tuned properly.

Tuning BGP Memory

To reduce or tune the BGP memory consumption, adjustments should be made to the three major factors leading to most BGP memory consumption:

  1. Prefixes
  2. Paths
  3. Attributes

Prefixes

BGP memory consumption becomes critical when BGP is holding a large number of prefixes or holding the Internet routing table. In most cases, not all the BGP prefixes are required to be maintained by all the routers running BGP in the network.

To reduce the number of prefixes, the following actions can be taken:

  1. Aggregation
  2. Filtering
  3. Partial routing table instead of full routing table

With the use of aggregation, multiple specific routes can be aggregated into one route. But aggregation is challenging when performed on a fully deployed running network.

Filtering provides control over the number of prefixes that should be maintained in the BGP table or advertised to BGP peers. BGP provides filtering based on prefix, BGP attributes, and communities. One important point to remember is that complex route filtering or route filtering applied for a large number of prefixes helps reduce the memory required but also requires additional CPU resources to apply the policy on BGP updates.

Many deployments do not require all the BGP speakers to maintain a full BGP routing table. The BGP speakers can maintain even a partial routing table, containing the most relevant and required prefixes. Such designs greatly reduce the resources being used throughout the network and increase scalability.

Managing the Internet RIB

If an enterprise is peering with an ISP, the ISP can advertise the full Internet routing table using BGP, generally, an enterprise doesn’t need access to the complete Internet routing table. If the ISP is advertising a full Internet table, there are few ways of managing the Internet routing table:

  • Ask the ISP to not send the whole Internet routing table by either filtering the needed routes or by advertising a default route.
  • If the ISP cannot filter the routes, filtering can be performed at the enterprise edge router for the required prefixes.
  • If resources conservation is the primary focus, ask the ISP to advertise a default route. This saves a lot of resources.

Example:

Figure shows how using the preceding three points scales the network and consumes fewer resources. The topology has three routers: R1, Internet-GW, and ISP-RTR. The Internet-GW router is the enterprise edge router peering with the ISP router named ISP-RTR.

  • For the sake of understanding, only fewer routes, say 50,000 routes, are being advertised by the ISP-RTR router.
  • Output illustrates that the BGP memory consumption on Internet-GW router is high with the ISP router advertising the Internet routing table. But after filtering those routes and configuring a default route, the memory utilization is much better as compared to what was before.
  • After the route filtering is performed and the static default route is configured toward the ISP, the command neighbor ip-address default-originate can be configured on the Internet-GW router toward the neighbor R1 to advertise the default route. This way, R1 has a path to reach the Internet.
				
					# Internet-GW
ip prefix-list DENY-IP seq 5 permit 0.0.0.0/0 le 32
!
route-map DENY-ALL deny 10
 match ip address prefix-list DENY-IP
!
router bgp 65000
 address-family ipv4 unicast
  neighbor 10.1.23.3 route-map DENY-ALL in
				
			

Paths

Sometimes the BGP table carries fewer prefixes but still holds more memory because of multiple paths. A prefix can be learned via multiple paths, but only the best or multiple best paths can be installed in the routing table.

To reduce the memory consumption by BGP due to multiple paths, the following solutions should be adopted:

  1. Reduce the number of peerings
  2. Use RRs instead of IBGP full mesh

Multiple BGP paths are caused by multiple BGP peers. Especially in an internal BGP (IBGP) full mesh environment, the number of BGP sessions increases exponentially, as does the number of paths. A lot of customers increase the number of IBGP neighbors to have more redundant paths, but two paths are sufficient to maintain redundancy. It is a well-known fact that IBGP needs to be in full mesh.

Attributes

A BGP route is a “bag” of attributes. Every BGP prefix has certain default or mandatory attributes that are assigned automatically, such as next-hop or AS-PATH or attributes that are configured manually, such as Multi-Exit Discriminator (MED), assigned by customers. Each attribute carried with the prefix contributes to the total amount of memory consumed. Along with attributes, communities – both standard and extended – add to increased memory consumption.

To reduce the BGP memory consumption due to various attributes and communities, the following solutions can be adopted:

  • Reduce the number of attributes
  • Filter standard or extended communities
  • Limit local communities
  • Reduce the newer types of attributes (that is, AIGP, IBGP PE-CE)

There is no method to get rid of the default BGP attributes, but the use of other elements can be controlled.

BGP community attributes make it easier to have more control over the BGP learned prefixes, both from Inter-AS and Intra-AS. Each destination can belong to multiple communities. Although there are no recommended ISP BGP communities apart from what is defined in RFC1998 or the four standard well-known communities, communities are very useful in an ISP network because it helps give control over the prefix to the customers. Communities are usually applied with route-maps.

Community attributes are numbers that represents specific meaning in the network. Based on the assigned value, certain actions are defined for the prefix in the network. As the network grows older and more mature, more communities are added over the period of time. Some of the older assigned community values lose their meaning or are not needed. But they are never touched, because network operators are afraid of breaking something in the network since they have no knowledge about those communities. Network administrators should try to reduce the number of BGP communities being used in the network. It not only makes the network much simpler to manage but also saves on network resources.

A router can receive a prefix with a community attribute attached and forward the community attached prefix to its peers, but the router controls the advertisement of the locally assigned community to another BGP peer using the command neighbor ip-address send-community. If the command is not specified, the locally assigned community is not advertised to BGP peers.

Leave a Reply

Related Post