VRF, MPLS and MPLS Layer 3 VPNs

Overlay Tunnels

A virtual private network (VPN) connects private networks together over a public network. Allows private networks to communicate with each other across an untrusted network such as the Internet. VPN data sent across an unsecure network needs to be encrypted to ensure that the data is not viewed or tampered with by an attacker. The most common VPN encryption algorithm used is IP Security (IPsec).

With VPNs, packets sent between private networks are encapsulated with new headers that are used to move the packets across the public network without exposing the private network’s original packet headers. This allows the packets to be forwarded between the two endpoints without any intermediary routers extracting information from the original packet headers and data. Once packets reach the remote endpoints, the VPN headers are removed, and the original headers are used to make forwarding decisions.

A VPN is a type of overlay network that exists on top of an existing network, known as the underlay network.

Examples of overlay tunneling technologies include the following:

  • Generic Routing Encapsulation (GRE)
    • IP Security (IPsec)
  • Locator ID/Separation Protocol (LISP)
  • Virtual Extensible LAN (VXLAN)
  • Multiprotocol Label Switching (MPLS)

MPLS tunneling is not supported across the Internet unless it is tunneled within another tunneling protocol, such as GRE, which can then be encrypted with IPsec (MPLS over GRE over IPsec).

VRF-Lite

  • VRF is a technology for creating separate virtual routers on a single physical router.
  • VRF-Lite provides VRF without MPLS.
  • Router interfaces, routing tables, and forwarding tables are isolated on an instance-by-instance basis and therefore prevent traffic from one VRF instance from interfering with another VRF instance.
  • With VRF-Lite, you can isolate the traffic within its respective virtual network and have multiple virtual routing tables on each router, each dedicated to its respective VRF instance.

By default, all router interfaces, the routing table, and any forwarding tables are associated with the global VRF instance. So, what you’ve been calling your routing table is actually the routing table of the global VRF instance. If you need to divide your router up into multiple virtual routers, you can do so by creating additional VRF instances; by doing so, you also create additional routing and forwarding tables.

MPLS Operation

Multiprotocol Label Switching (MPLS) is a packet-forwarding method that makes forwarding decisions based on labels instead of on the Layer 3 destination of the packet.

MPLS is not much faster than traditional IP routing. So why would you even consider MPLS? Well, MPLS decreases forwarding overhead on core routers, making them more efficient. In addition, MPLS can forward other Layer 3 protocols besides IPv4, and MPLS supports multiple services, such as unicast routing, multicast routing, VPNs, Traffic Engineering (TE), QoS, and Any Transport Over MPLS (AToM). Therefore, MPLS is very efficient and flexible.

When a router receives an IP packet, it looks at the destination IP address, checks its routing table to figure out how to forward it, and sends the packet to the next router which does the exact same thing until we reach the destination.

Routers support longest prefix matching. This means the router will use the most specific prefix in the RIB. Finding the longest prefix in the routing table is compute-intensive.

To reduce the burden of looking up destinations in the RIB, routers typically looked up the destination the first time they received an IP packet in software and used hardware to process other packets that belong to the same flow.

Each router makes independent decisions.

When you transmit an IP packet from SRC to DST using the Internet, there are no guarantees when it comes to availability, bandwidth, and delay. It’s all “best-effort”.

Main reason for MPLS was to reduce the amount of IP routing lookups. Back then, hardware wasn’t as powerful as it is today.

Exact lookups are much easier to implement in hardware. The idea behind MPLS is to “switch” based on labels with lookups that use exact matching instead of the compute-intensive longest prefix IP routing lookups.

Each prefix in the routing table gets a label and routers exchange labels with each other, establishing a label-switched path (LSP). Instead of lookups in the routing table, routers now “switch” packets based on the labels.

MPLS LIB and LFIB

The control plane of the MPLS-enabled router is responsible for exchanging labels with other MPLS-enabled routers, using a label distribution protocol, in addition to exchanging routing information using routing protocols to populate the IP routing table (RIB).

Once labels have been exchanged, the label information is used to populate the LIB, and then the best label information can be used to populate the Label Forwarding Information Base (LFIB).

Control and Data Plane of an LSR
  1. An unlabeled IP packet arrives with a destination of 10.0.0.5.
  2. Because it is unlabeled, the FIB is used to make a forwarding decision.
    • If the FIB indicates that the outgoing interface is not an MPLS-enabled interface, the packet is forwarded unlabeled.
    • If the FIB indicates that the outgoing interface is an MPLS-enabled interface, a label is added to the packet, and the labeled packet is forwarded, labeled, out the MPLS interface.

  3. A labeled packet arrives on an MPLS-enabled interface.
  4. Because it is labeled, the LFIB is used to make a forwarding decision.
    1. If the LFIB indicates that the outgoing interface is an MPLS-enabled interface, the label is removed, a new label is added, and the labeled packet is forwarded out the MPLS interface, labeled.
    2. If the LFIB indicates that the outgoing interface is not an MPLS-enabled interface, the label is removed, and the unlabeled packet is forwarded, unlabeled, using the information in the FIB.

Label Switching Routers (LSR)

  • R1 through R5 are part of the MPLS domain. They are known as label switching routers (LSRs) because they support MPLS. They understand MPLS labels and can receive and transmit labeled packets on their interfaces.

  • R1 and R5 are considered edge LSRs.
    • An edge LSR sits at the edge of the MPLS domain, adds labels to packets that are entering the MPLS domain (known as an ingress LSR), removes labels from packets that will be leaving the MPLS domain (known as an egress LSR), and even forwards packets as needed based on labels or the lack of a label.

  • R2, R3, and R4 are considered intermediate LSRs.
    • An intermediate LSR sits within the MPLS domain and primarily forwards packets using label information.

Label-Switched Path (LSP)

The label-switched path (LSP) is the cumulative labeled path (sequence of routers) that a labeled packet takes through the MPLS domain.

It is a unidirectional path, therefore, in a complex network with multiple potential paths between source and destination, it is possible that the LSP from source to destination could be different from the LSP that is used for the return traffic.

However, typically the same path in reverse is used for the return traffic because of the underlying dynamic routing protocols, such as OSPF and EIGRP, that are used to build the symmetrical network and its forwarding paths.

  • In this case, the LSP from R1 to 10.0.0.0/24 uses labels 87, 11, 65, and 23.
  • Along the path, each router examines the label to make a forwarding decision, removes the label, adds a new label if required, and then forwards the packet.
Label-Switched Path in an MPLS Domain

Labels

  • For MPLS to work, a label needs to be added to the packet.
  • The label is added as a shim header between the Layer 2 frame header and the Layer 3 packet header.
  • The label is 4 bytes (32 bits) in size and contains four different fields.
    • The first 20 bits (label) define the label number, the next 3 bits (EXP) are used for quality of service (QoS), the 1 bit (S) field is used to define whether the label is the last label in the stack when more than one label is used in the packet (for example, with MPLS VPNs), and the final 8 bits (TTL [Time To Live]) is used just like IP’s TTL so that MPLS frames are discarded if they have not reached the destination by the time the TTL reaches 0.

MPLS-enabled routers automatically assign labels to every network that they know about. How does a router know about a network? It can be locally configured by configuring an IP address on a router interface and issuing the no shutdown command on the interface or through the propagation of routing information with dynamic routing protocols such as OSPF and EIGRP.

Example:

  • R5 gave a label of 23 to network 10.0.0.0/24.
  • R4 gave a label of 65 to network 10.0.0.0/24.
  • R3 gave a label of 11 to network 10.0.0.0/24.
  • R2 gave a label of 87 to network 10.0.0.0/24.
  • R1 gave a label of 19 to network 10.0.0.0/24.

What you should notice from this is the local significance of labels. Each router, regardless of whether it is locally connected to the network 10.0.0.0/24, like R5, or not locally connected, like the other routers, generates a local label for the network it knows about, regardless of how it learned about it.

Routers associating a Label for the 10.0.0.0/24 network

Label Distribution Protocol (LDP)

  • In order to build the LSP, labels need to be shared/distributed with directly connected LSRs. This is done using a label distribution protocol such as Label Distribution Protocol (LDP), which is the most common protocol in use when sharing/distributing labels for IPv4 prefixes.
  • Once MPLS has been enabled on an interface, LDP hello packets are sent out the interface to the destination multicast address 224.0.0.2 (the all routers multicast address), using UDP port 646.
  • Any device on that same link that is also enabled for MPLS and that receives the hello packet forms an LDP TCP session using port 646 with the neighboring device so that label information can be exchanged.
  • Within the hello packet is an LDP ID that is used to uniquely identify the neighbor and the label space, which will either be per platform (same label used out all interfaces for a single destination) or per interface (different label used out each interface for a single network).
  • When establishing the LDP TCP session between two LSRs, one of the routers needs to be the active router. The active router is responsible for setting up the TCP session. The router with the higher LDP ID is selected as the active router and sets up the TCP session between the two routers.

Example: Shows how..

  • R1 distributes its label of 19 for network 10.0.0.0/24 out all MPLS-enabled interfaces.
  • R2 distributes its label of 87 for network 10.0.0.0/24 out all MPLS-enabled interfaces.
  • R3 distributes its label of 11 for network 10.0.0.0/24 out all MPLS-enabled interfaces.
  • R4 distributes its label of 65 for network 10.0.0.0/24 out all MPLS-enabled interfaces.
  • R5 distributes its label of 23 for network 10.0.0.0/24 out all MPLS-enabled interfaces after the TCP sessions have been established.
LSRs using LDP to Distribute Lables out all MPLS-Enabled Interfaces

Each router takes the labels it has learned from the LDP neighbors and populates its LIB. At first, this does not look right because if you focus on R3, there are three entries for 10.0.0.0/24. What does each one mean?

  • The entry with label 11 is the locally significant label. It is the label R3 advertises to other routers, so they know which label to place in a packet when they send a packet to R3 that is destined to 10.0.0.0/24.
  • The entry with the label 87 is from R2, and it is the label that R2 wants R3 to use when R3 sends to R2 packets that are destined to 10.0.0.0/24.
  • The entry with the label 65 is from R4, and it is the label that R4 wants R3 to use when R3 sends to R4 packets that are destined to 10.0.0.0/24.

What we are examining is the LIB, which contains all the labels that the router knows about for all the different destination networks it knows about. It then takes the best labels/networks and populates the LFIB, which is used to make forwarding decisions.

Label Switching

Based on information found in the IP routing table and the LIB, the routers have populated their LFIB and FIB as shown:

  • When R1 receives a packet with the label 19, it examines the LFIB and notices that it must forward it with the label 87 to R2.
    • However, in this case, if the packet is coming in the interface that is connected to the 192.168.0.0/24 network, it does not have a label as there are no routers imposing labels on packets out that interface. Therefore, when the packet arrives on the interface connected to the 192.168.0.0/24 network of R1 with a destination IP address in the packet for 10.0.0.0/24, the FIB indicates that label 87 has to be added, and the packet needs to be forwarded with R2 as the next-hop address.
  • When R2 receives the packet with the label 87, it examines the LFIB and notices that it must forward it with the label 11 to R3 (as the next-hop address).
  • When R3 receives the packet with the label 11, it examines the LFIB and notices that it must forward it with the label 65 to R4 (as the next-hop address).
  • When R4 receives the packet with the label 65, it examines the LFIB and notices that it must forward it with the label 23 to R5 (as the next-hop address).
  • When R5 receives the packet with the label 23, it examines the LFIB and notices that there is no outgoing label, which means it is the end of the LSP, and therefore the label must be removed, and normal routing has to occur with the IP routing table (FIB).
Populated FIBs and LFIBs

Penultimate Hop Popping (PHP)

R5 must do two lookups when it receives a labeled packet destined to 10.0.0.0/24.

  • First, it must look in the LFIB because it received a labeled frame. In this case, there is no label out. Therefore, it must remove the label and make a forwarding decision based on a second lookup, using the FIB to forward the packet. This is not efficient.

The solution to this inefficiency is penultimate hop popping (PHP). With PHP, R4 pops the label before sending the packet to R5.

  • So, instead of R5 advertising a label of 23 to R4 for 10.0.0.0/24, as in the previous scenarios, it advertises a “pop.”
  • Essentially, R5 tells R4 that it is the end of the LSP for the 10.0.0.0/24 network, and that R4 should remove any label and forward the packet, unlabeled, to R5.
    • Therefore, R5 receives an unlabeled packet and can do a single lookup using the FIB to forward the packet.
  • R5 has advertised the label “pop” to R4, and R4 has populated its LFIB accordingly. Notice the “pop” in the Label Out column of R4’s LFIB.
    • Now, when R4 receives a packet with the label 65, it pops the label and forwards it through R5.
  • R5 receives the unlabeled packet and uses the FIB to forward the packet to the destination IP address in network 10.0.0.0/24.
PHP: R5 indicating to R4 to Pop the Label

MPLS Layer 3 VPN

MPLS Layer 3 VPNs provide peer-to-peer connectivity between private customer sites across a shared network, with Customer A and Customer B both using the same MPLS domain to connect their own private sites together.

  • In an MPLS Layer 3 VPN architecture, customer routers are known as CE (customer edge) routers, and they do not run MPLS. In fact, they have no knowledge of MPLS, labels, or even VRF instances, which makes it easier for the customer to take advantage of the benefits provided by a provider’s MPLS domain.
  • The CE routers connect to the PE (provider edge) routers of the MPLS domain. The PE routers, such as PE_R1 and PE_R5 are the ingress and egress LSRs for the MPLS domain.
  • The P (provider) routers, such as P_R2, P_R3, and P_R4, are the intermediate LSRs of the MPLS domain.

The goal is to have Customer A Site 1 and Customer A Site 2 exchange their local routing information over the MPLS domain and then forward traffic as needed from Site 1 and Site 2 over the MPLS domain. The same would be true for Customer B Site 1 and Customer B Site 2.

Due to the nature of the MPLS Layer 3 VPN, overlapping address spaces between customers is of no concern. Therefore, Customer A and Customer B can be using the same private IP address space.

Once the PE routers learn routes from the CE routers, the PE routers redistribute the routes into MP-BGP so they can be exchanged with other PE routers. When another PE router receives the routes, they are redistributed into an IGP and placed in the correct customer VRF instance so they can be exchanged with the CE router

An important point to consider is that the P routers are not participating in BGP. Only the PE routers are. They are forming an MP-IBGP neighborship with each other and exchanging the routes using the underlying network that is built with an IGP such as OSPF or IS-IS. So the PE routers and the P routers are using a dynamic routing protocol to learn about all the destinations in the P network, and only the PE routers are using MP-IBGP on top of that to exchange the customer routes.

VPNv4 Address

Let’s now go back to overlapping IPv4 address spaces.

If all customer routes are being redistributed into MP-BGP, how does BGP handle identical network prefixes that belong to different customers? It uses a route distinguisher (RD) to expand the customer’s IP prefix so that it includes a unique value that distinguishes it from the other identical prefixes. The RD is generated and used by the PE routers on a per-customer VRF instance basis, and to keep things simple, the RD is used regardless of whether there are overlapping address spaces. So, the RD is used all the time.

The unique 64-bit RD is prepended to the 32-bit customer prefix (IPv4 route) to create a 96-bit unique prefix called a VPNv4 address. This VPNv4 address is exchanged by the MP-IBGP neighboring routers.

VPNv4 Address Format

Example:

  • The Customer A VRF instance is using an RD of 1:100, and the Customer B VRF instance is using an RD of 1:110.
  • When these RDs are prepended to the IPv4 prefixes, the results are a VPNv4 route of 1:100:192.168.0.0/24 and a VPNv4 route of 1:110:10.0.0.0/24.
  • Now let’s say Customer A also chooses to use the 10.0.0.0/24 network and advertise it over to its other site. This is where the RDs keep everything unique.
  • Customer A would have a VPNv4 route of 1:100:10.0.0.0/24, and Customer B would have the route 1:110:10.0.0.0/24.
  1. The CE router and PE router exchanges routes using a dynamic routing protocol such as OSPF or EIGRP.
  2. The PE router places the customer-specific routes in the customer-specific VRF table.
  3. The routes in the customer’s VRF table are redistributed into MP-BGP as VPNv4 routes.
  4. The PE routers exchange VPNv4 routes over their MP-IBGP peering.
  5. The PE router redistributes the VPNv4 routes as OSPF, EIGRP, and so on routes into the customer-specific VRF table.
  6. The PE router and CE router exchange routes using a dynamic routing protocol such as OSPF or EIGRP.

VPN Label Stack

For the MPLS domain to forward traffic, a label stack is required. Specifically, two labels are required for traffic to be successfully forwarded through the MPLS domain.

  • The first label that is attached to the packet is a VPN label.
  • The second label that is attached is the LDP label.
  • When the IP packet arrives at the ingress PE router, the PE router attaches both labels.
  • The egress router uses the VPN label to determine customer specifics about the packet and what should be done with it. The LDP label is used for label switching from PE to PE in the MPLS domain.
  • VPN labels are learned from PE routers over the MP-IBGP peering.
  • LDP labels are learned using the LDP protocol.

Example: VPN Label

Let’s say that..

  • When PE_R5 learns of 10.0.2.0/24 from CE_RB, it places it in the Customer B VRF instance. It then redistributes it into MP-BGP and thus creates a VPNv4 route of 1:110:10.0.2.0/24.
  • This VPNv4 route needs a VPN label created for it so that forwarding will be successful. In this case, PE_R5 assigns it the label 35. This label is shared with PE_R1 over the MP-IBGP peering they have.
  • Now any time PE_R1 receives an IP packet that is destined for 10.0.2.0/24, it knows to attach the label 35 so the packet can be forwarded.
  • However, this label is known only by the PE routers. Therefore, if PE_R1 forwards this VPN packet to P_R2, it will be dropped as it has no idea what the VPN label 35 means. Therefore, the LDP label is needed to forward the packet from PE_R1 and PE_R5.
VPN Label assgined by PE_R5 and shared with PE_R1

Example: LDP Label

Shows how LDP is used to exchange labels that have been generated by the PE routers (ingress and egress LSRs) and the P routers (intermediate LSRs).

  • PE_R5 tells P_R4 to pop the label.
  • P_R4 tells P_R3 to use the label 52.
  • P_R3 tells P_R2 to use the label 10.
  • P_R2 tells PE_R1 to use the label 99.
LDP Label Assigment

Example: LSP (Label-Switched Path)

The complete LSP is now ready to label switch the VPN packet from PE_R1 to PE_R5.

MPLS Layer 3 VPN Label-Switched Path

Packet forwarding thru the MPLS L3 VPN

  1. When an IP packet destined to 10.0.2.0/24 arrives at PE_R1 from CE_RA, PE_R1 determines that the packet needs a VPN label of 35 so PE_R5 will know what to do with the VPN packet and an LDP label of 99 so that the VPN packet can be label switched through the MPLS domain.
  2. Once the label stack is complete, PE_R1 sends the label-stacked packet to P_R2.
    • When P_R2 receives it, it only examines the LDP label. Based on the LFIB, it states that the label 99 needs to be swapped to 10 and forwarded to P_R3. So, it does so.
    • When P_R3 receives it, it only examines the LDP label. Based on the LFIB, it states that the label 10 needs to be swapped to 52 and forwarded to P_R4. So, it does so.
    • When P_R4 receives it, it only examines the LDP label. Based on the LFIB, it states that the label 52 needs to be popped and forwarded to P_R5. So, it makes this happen.
  3. Now PE_R5 only needs to read the VPN label, which is 35. The label is removed, and the VRF instance for Customer B is used to forward the IP packet to the CE_RB router.
Forwarding through the MPLS L3 VPN Domain

Leave a Reply

Related Post

MPLS OverviewMPLS Overview

Unicast IP Forwarding in Traditional IP Networks In traditional IP networks, routing protocols are used to distribute Layer 3 routing information. Regardless of the routing protocol, packet forwarding is based

MPLS QAMPLS QA

MPLS Technology Basics P (Provider) router = Label Switching Router (LSR) Runs an IGP and LDP PE (Provider Edge) router = edge router (LER) Imposes and removes MPLS labels Runs