Introduction to MPLS
MPLS technology is being widely adopted by service providers worldwide to implement VPNs to connect geographically separated customer sites.
- MPLS VPNs are Layer 3 WAN solution to an age-old Layer 2 WAN problem, to provide any-to-any connectivity among sites in a cost efficient manner.
- With MPLS we can have a Layer 3 fully meshed network.
- VPNs allow the use of a shared infrastructure offered by an ISP to implement private networks. Doesn’t necessary mean confidentiality and/or integrity.
- Overlay VPNs – service providers provide virtual point-to-point links.
- Peer-to-Peer VPNs – service providers participate in the customer routing.
- Runs an IGP
- Not MPLS aware
- High end router.
- Connects to both CE and P routers, distribute VPN information via MP-BGP to other PE with VPNv4 addresses, extended community and label.
- Each customer is assigned its own RD and VRF table dedicated to maintain routing information.
- Routing across backbone is performed by another routing process using the global RIB.
- Single router but runs multiple instances of an IGP – one for each customer.
- Multiple instances of IGP are redistributed into a global routing table.
- VRF – a customer specific RIB
- Provides isolation between customer routes
- Information from VRF is exchanged between PE routers.
- A single routing protocol is used between PE routers to exchange customer routes without the involvement of the P routers (MP-BGP).
- BGP the only real protocol of choice for the provider – scalability (very large routing tables)
- Number of prefixes advertised by each customer
- BGP peering are configured between PE routers directly so that prefixes can be exchanged for a given customer.
- Provide transport for traffic between PEs.
- P and PE routers shared a common IGP.
- Participate in LDP.
- Forward packets by looking at labels.
- The ISP only offers Internet connectivity and no other services. Each customer uses the ISP to have connectivity between their sites.
- To accomplish this, the ISP is running eBGP between the CE and PE routers to exchange prefixes. This means all internal (P) routers of the ISP have to run iBGP or they don’t know where to forward their packets to.
- A full internet routing table currently has > 500.000 prefixes and with 8 ISP routers running iBGP, we need 28 iBGP peerings. All routers have to do lookups in the routing table for any possible destination.
- Now the goal is to have connectivity between two customer sites, why should all internal P routers know about this? The only routers that need to know how to reach the customer sites are the PE routers. Why not build a tunnel between the PE routers?
- The two Top PE routers will use a GRE tunnel for the customer A sites.
- The two bottom PE routers will use a GRE tunnel for the customer B sites.
- With this, we have a BGP free core.
- Only two places where BGP is needed:
- eBGP between PE and CE
- iBGP between PE routers
Why do we need MPLS?
We used a GRE tunnel but we could have used any tunneling mechanism. Besides GRE, there’s IP-in-IP, Q-in-Q and…..MPLS
MPLS (Multi Protocol Label Switching)
- Multi Protocol: besides IP, we can tunnel anything, IP, IPv6, Ethernet, PPP, etc.
- Label Switching: forwarding is done based on labels, not destination based routing.
MPLS VPN Architecture and Terminology
The main components of MPLS VPN architecture are:
- Customer Network: which is usually a customer-controlled domain consisting of devices or routers spanning multiple sites belonging to the customer.
- CE routers: which are routers in the customer network that interface with the service provider network.
- Provider network: which is the provider-controlled domain consisting of provider edge and provider core routers that connect sites belonging to the customer on a shared infrastructure. The provider network controls the trafﬁc routing between sites belonging to a customer along with customer trafﬁc isolation.
- PE routers: which are routers in the provider network that interface or connect to the customer edge routers in the customer network.
- P routers: which are routers in the core of the provider network that interface with either other provider core routers or provider edge routers.
MPLS VPN Routing Model
From a CE router’s perspective, only IPv4 updates, as well as data, are forwarded to the PE router. The CE router does not need any specific configuration to enable it to be a part of a MPLS VPN domain. The only requirement on the CE router is a routing protocol (or a static/default route) that enables the router to exchange IPv4 routing information with the connected PE router.
P / PE Routers
- The PE router must first be capable of isolating customer traffic if more than one customer is connected to the PE router. Each customer, therefore, is assigned an independent routing table similar to a dedicated PE router in the initial peer-to-peer discussion.
- Routing across the SP backbone is performed using a routing process in the global routing table.
- P routers provide label switching between provider edge routers and are unaware of VPN routes.
- CE routers in the customer network are not aware of the P routers and, thus, the internal topology of the SP network is transparent to the customer.
- The P routers are only responsible for label switching of packets. They do not carry VPN routes and do not participate in MPLS VPN routing.
- The PE routers exchange IPv4 routes with connected CE routers using individual routing protocol contexts. To enable scaling the network to large number of customer VPNs, multiprotocol BGP is configured between PE routers to carry customer routes.
VRF: Virtual Routing and Forwarding Table
- Customer isolation is achieved on the PE router by the use of virtual routing tables (VRFs).
- In essence, it is similar to maintaining multiple dedicated routers for customers connecting into the provider network.
- The function of a VRF is similar to a global routing table, except that it contains all routes pertaining to a specific VPN versus the global routing table.
- The VRF also contains a VRF-specific CEF forwarding table analogous to the global CEF table.
- The interface that is part of the VRF must support CEF switching.
- The number of interfaces that can be bound to a VRF is only limited by the number of interfaces on the router, and a single interface (logical or physical) can be associated with only one VRF.
Cisco IOS supports a variety of routing protocols as well as individual routing processes (OSPF, EIGRP, etc.) per router. However, for some routing protocols, such as RIP and BGP, IOS supports only a single instance of the routing protocol. Therefore, to implement per VRF routing using these protocols that are completely isolated from other VRFs, which might use the same PE-CE routing protocols, the concept of routing context was developed.
Route Distinguisher, Route Targets, MP-BGP, VPN Label, and Address Families
In the MPLS VPN routing model, the PE router provides isolation between customers using VRFs. However, this information needs to be carried between PE routers to enable data transfer between customer sites via the MPLS VPN backbone. The PE router must be capable of implementing processes that enable overlapping address spaces in connected customer networks. The PE router must also learn these routes from attached customer networks and propagate this information using the shared provider backbone. This is done by the association of a route distinguisher (RD) per virtual routing table on a PE router.
A RD is a 64-bit unique identifier that is prepended to the 32-bit customer prefix or route learned from a CE router, which makes it a unique 96-bit address that can be transported between the PE routers in the MPLS domain. Thus, a unique RD is configured per VRF on the PE router. The resulting address, which is 96-bits total (32-bit customer prefix + 64-bit unique identifier or RD), is called a VPN version 4 (VPNv4) address.
VPNv4 addresses are then exchanged between PE routers in the provider network. Example shows the same IP prefix 172.16.10.0/24, received from two different customers, is made unique by prepending different RD values, 1:100:1 and 1:101, prior to propagating the addresses as VPNv4 addresses on the PE router.
The protocol used for exchanging these VPNv4 routes between PE routers is multiprotocol BGP (MP-BGP). BGP capable of carrying VPNv4 (96-bit) prefixes in addition to other address families is called MP-BGP.
The IGP requirement to implement iBGP (internal BGP) still holds in the case of an MPLS VPN implementation. Therefore, the PE router must run an IGP that provides NLRI information for iBGP if both PE routers are in the same AS. Cisco currently supports both OSPFv2 and ISIS in the MPLS provider network as the IGP.
MP-BGP is also responsible for assignment of a VPN label. Packet forwarding in an MPLS VPN mandates that the router specified as the next hop in the incoming BGP update is the same router that assigns the VPN label. Scalability was a primary reason for the choice of BGP as the protocol to carry customer routing information.
An MP-BGP session between PE routers in a single BGP AS is called an MP-iBGP session.
Route targets (RTs) are additional identifiers used in the MPLS VPN domain in the deployment of MPLS VPN that identify the VPN membership of the routes learned from that particular site. RTs are implemented by the use of extended BGP communities in which the higher order 16 bits of the BGP extended community (64 total bits) are encoded with a value corresponding to the VPN membership of the speciﬁc site. When a VPN route learned from a CE router is injected into VPNv4 BGP, a list of VPN route target extended community attributes is associated with it.
- The export route target is used in identiﬁcation of VPN membership and is associated to each VRF. This export route target is appended to a customer preﬁx when it is converted to a VPNv4 preﬁx by the PE router and propagated in MP-BGP updates.
- The import route target is associated with each VRF and identiﬁes the VPNv4 routes to be imported into the VRF for the speciﬁc customer. The format of a RT is the same as an RD value.
The following processes occur during route propagation in an MPLS VPN:
- The prefix 172.16.10.0/24 is received from CE1-A, which is part of VRF CustomerA on PE1-AS1.
- PE1 associated an RD value of 1:100 and an export RT value of 1:100 as configured in the VRF definition.
- Routes learned from connected CE routers CE1-A are redistributed into the MP-BGP process on PE1-AS1 where the prefix 172.16.10.0/24 is prepended with the RD value of 1:100 and appended with the route target extended community value (export RT) of 1:100 prior to sending the VPNv4 prefix as part of the MP-iBGP update between PE routers.
- The VPN label (3 bytes) is assigned for each prefix learned from the connected CE router’s IGP process within a VRF by the PE router’s MP-BGP process.
- MP-BGP running in the service provider MPLS domain thus carries the VPNv4 preﬁx (IPv4 preﬁx + prepended RD) in addition to the BGP route target extended community.
- Note that although the RT is a mandatory conﬁguration in an MPLS VPN for all VRFs conﬁgured on a router, the RT values can be used to implement complex VPN topologies in which a single site can be a part of more than one VPN.
- The VPN label is only understood by the egress PE (data plane) that is directly connected to the CE router advertising the prefix.
- The MP-BGP update is received by the PE router PE2, and the route is stored in the appropriate VRF table for Customer A based on the VPN label.
- The received MP-BGP routes are redistributed into the VRF PE-CE routing processes, and the route is propagated to CE2-A.
In addition, other BGP extended community attributes such as site of origin (SoO) can also be applied to the MP-iBGP update prior to propagation. The SoO attribute is used to identify the speciﬁc site from which the PE learns the route and is used in the identiﬁcation and prevention of routing loops. The SoO extended community is a BGP extended community attribute used to identify routes that have originated from a site so that the re-advertisement of that prefix back to the source site can be prevented, thus preventing routing loops. The SoO extended community uniquely identifies the site from which a PE router has learned a route. SoO enables filtering of traffic based on the site from which it was originated. SoO filtering manages MPLS VPN traffic and prevents routing loops from occurring in complex and mixed network topologies in which the customer sites might possess connectivity across the MPLS VPN backbone as well as possess backdoor links between sites.
It is important to understand the concept of address families and their place in the operation of MP-BGP to enable the transport of VPNv4 routes with extended community attributes. Prior to RFC 2283, “Multiprotocol Extensions for BGP-4,” BGP version 4 was capable of carrying routing information only pertaining to IPv4. RFC 2283 deﬁnes extensions to BGP-4 that enable BGP-4 to carry information for multiple network layer protocols. RFC 2283 states that to enable BGP-4 to support routing for multiple network layer protocols, the additions to BGP-4 must account for the ability of a particular network layer protocol to be associated with a next hop as well as the NLRI (network layer reachability information). The two new attributes that were added to BGP were Multiprotocol Reachable NLRI (MP_REACH_NLRI), and Multiprotocol Unreachable NLRI (MP_UNREACH_NLRI). MP_REACH_NLRI carries the set of reachable destinations together with the next-hop information to be used for forwarding to these destinations. MP_UNREACH_NLRI carries the set of unreachable destinations. Both of these attributes are optional and nontransitive. Therefore, a BGP speaker that does not support these multiprotocol capabilities will just ignore the information carried in these attributes and will not pass it to other BGP speakers.
The PE router, in essence, is an Edge LSR and performs all the functions of an Edge LSR.
- The PE router requires LDP for label assignment and distribution as well as forward labeled packets.
- PE implements a routing protocol (or static routes) with connected CE routers per virtual routing table and requires MP-BGP to propagate prefixes learned from CE routers as VPNv4 prefixes in MP-iBGP updates to other PE routers along with the VPN label.
The P router’s requirements are to run an IGP (either OSPF or ISIS) as well as have MPLS enabled to forward labeled packets (data plane) between PE routers. The IGP is used to provide, as well as propagate, NLRI to connected P and PE routers to implement an MP-iBGP session between PE routers (control plane).
MPLS VPN Control Plane Operation
The control plane in MPLS VPN implementation contains all the Layer 3 routing information and the processes within to exchange reachability information for a specific Layer 3 IP prefix in addition to label assignment and distribution using LDP.
The data plane performs the functions relating to the forwarding of both labeled as well as IP packets to the next hop toward a destination network.
- The CE routers are connected to the PE routers, and an IGP, BGP, or static route is required on the CE routers in conjunction with attached PE routers to gather and advertise NLRI information.
- In the MPLS VPN backbone consisting of P and PE routers, an IGP (usually either OSPF or ISIS) in addition to LDP is used between PE and P routers. LDP is used for allocation as well as distribution of labels in the MPLS domain. The IGP is used for NLRI information exchange.
- MP-BGP sessions are maintained between PE routers in an MPLS VPN domain and exchange MP-BGP updates consisting of unique VPNv4 addresses in addition to BGP extended community attributes associated with specific VPNv4 addresses.
- Packets from CE to PE are always propagated as IPv4 packets.
The following are the steps for control plane operation in MPLS VPN. The steps are outlined for preﬁxes advertised by the CEA-1 router:
Step 1: IPv4 update for network 172.16.10.0 is received by the egress PE router (data plane).
Step 2: PE1-AS1 accepts and transforms the IPv4 route, 172.16.10.0/24, to a VPNv4 route by assigning an RD 1:100, SoO, and RT 1:100 based on the VRF configuration on PE1-AS1. It then allocates a VPNv4 label V1 to the 172.16.10.0/24 update and rewrites the next-hop attribute to the PE1-AS1 loopback0 IP address 10.10.10.101. PE1-AS1 loopback 10.10.10.101 is reachable via IGP (OSPF) and LDP.
This propagation takes place as soon as the MPLS VPN provider network is established.
The following steps are performed in the label propagation process for prefix 10.10.10.101/32. This operation is shown for clarity:
- 2a: Edge LSR PE2-AS1 requests a label for the 10.10.10.101/32 preﬁx using the LDP label mapping request from its downstream neighbor, LSR P2-AS1.
P2-AS1 requests a label for the 10.10.10.101/32 preﬁx using the LDP label mapping request from its downstream neighbor LSR P1-AS1.
P1-AS1, in turn, requests a label for the 10.10.10.101/32 preﬁx using the LDP label mapping request from its downstream neighbor, Edge LSR PE1-AS1.
Edge LSR PE1-AS1 allocates a label of implicit-null (penultimate hop popping) to 10.10.10.101/32, modiﬁes the entry in the LFIB corresponding to 10.10.10.101/32, and sends it to P1-AS1 using an LDP reply.
- 2b: P1-AS1 uses the implicit-null label received from PE1-AS1 as its outbound label value, allocates a label (L1) to preﬁx 10.10.10.101/32, and modiﬁes the LFIB entry for 10.10.10.101/32. P1-AS1 then sends this label value to P2-AS1 via an LDP reply.
- 2c: P2-AS1 uses the label (L1) received from P1-AS1 as its outbound label value, allocates a label (L2) to preﬁx 10.10.10.101/32, and modiﬁes the LFIB entry for 10.10.10.101/32. P2-AS1 then sends this label value to PE2-AS1 via an LDP reply.
Step 3: PE2-AS1 has the VRF configured to accept routes with RT 1:100 and therefore translates the VPNv4 update to IPv4 and inserts the route in VRF for Customer A. It then propagates this route to the CE2-A.
MPLS VPN Data Plane Operation
MPLS VPN data plane operation involves the usage of the label stack in which the top label in the label stack is the label assigned for the egress PE routers (data plane) next-hop address, and the second label in the label stack is the VPN label assigned by the egress PE router connected to the CE router advertising the prefix.
P routers perform label switching on the LDP-assigned label toward the egress PE router. The egress PE router identifies the VPN label assigned with a VRF (that it has previously assigned) and either forwards the IP packet toward the CE router or performs another IP lookup in the VRF table to identify the next hop toward the destination.
When data is forwarded to a specific prefix belonging to a VPN across the MPLS-enabled core, the top label in the label stack is the only one swapped as the packet traverses the backbone. The VPN label is kept intact and is removed only in the egress/downstream PE router. The resulting prefix is associated with an outgoing interface belonging to a specific VRF on the router depending on the value in the VPN label.
Here are the steps in the data plane forwarding:
- CE2-A originates a data packet with the source address of 172.16.20.1 and destination of 172.16.10.1.
- PE2-AS1 receives the data packet and appends the VPN label V1 and LDP label L2 and forwards the packet to P2-AS1.
- P2-AS1 receives the data packet destined to 172.16.10.1 and swaps LDP label L2 with L1.
- P1-AS1 receives the data packet destined to 172.16.10.1 and pops the top label because it receives an implicit-null label mapping for 10.10.10.101/32 from PE1-AS1. The resulting labeled packet (with VPN Label V1) is forwarded to PE1-AS1.
- PE1-AS1 pops the VPN label and forwards the data packet to CE1-A where the 172.16.10.0 network is located.
The key to understanding the operation of MPLS VPN is that the VPN label is never touched until it reaches the egress PE router toward the FEC.
The next-hop label mapping to the downstream PE router’s loopback is used to forward the packet (in this case, labeled IP because of the presence of a VPN label) through the MPLS domain.