Where does SD WAN over MPLS fit in today’s networks?
With the proliferation and maturing of different major SD WAN platforms, many vendors make the claim that you can completely remove your old, expensive Multi-Protocol Label Switching (MPLS)-based circuits and replace them with cheaper commodity Internet circuits.
Many SD WAN vendors will suggest SD WAN is an Internet only technology. This kind of marketing is misleading, software WAN services are agnostic with regards to capability, terminating both public IP and private MPLS.
Why would you continue to pay the extra price for using private MPLS circuits? Why would you continue to pay the extra price for using private MPLS circuits when SD WAN over the Internet can save serious money? Several industry pundits feel the same way. Their argument is that the proprietary technologies, or “special sauce”, contained within each SD WAN platform overcomes previously-perceived limitations of using the public Internet for your wide area network (WAN) transport. Why would you continue to pay the extra price for using private MPLS circuits? How does connectivity to modern cloud-based services factor into the equation?
From leased lines to MPLS
The need to connect remote offices and Data Centres together has existed for decades, and the technologies used for the interconnections have evolved over the years. The amount of bandwidth available over the interconnections has increased, and the cost has decreased. The increase in bandwidth and correlated decrease in price is directly tied to telecommunications carriers evolving their core infrastructure from dedicated circuit-switched technologies to statistically oversubscribed packet-switched technologies like MPLS.
From the 1980s into the early 2000s, leased lines were the most popular form of WAN circuit technology. Leased lines were typically built with Asynchronous Transfer Mode (ATM) or Frame Relay as the underlying technology, though technologies such as X.25, ISDN BRI (basic rate interface) and PRI (primary rate interface), and others were also used. Locations were interconnected with dedicated, private virtual circuits through the carrier’s network.
They were referred to as virtual circuits because their path could change within the carrier’s network based on logical configuration, but were still considered dedicated because the entire bandwidth provisioned on the virtual circuit was available for consumption at all times by the customer organisation. These were strictly Layer 2 connections, which meant it was entirely up to the consuming organisation to establish their own routing over the circuits.
Then in the late 1990s and into the 2000s, MPLS was developed as a technology designed to save money in service provider networks through various applications. The original motivation behind MPLS was to provide a service similar to ATM, but over IP networks. ATM networks establish a switched path through the network before transmission of data occurs, with certain values in the ATM cell header (VPI/VCI) changed at every intermediate ATM switch. MPLS operates similarly by establishing a Label-Switched Path (LSP) before data transmission occurs, with the label swapped for a different value at each intermediate forwarding device (also known as a “hop”) along the path from source to destination. Label-swapping along a predetermined path is much less resource intensive than what is required for traditional IP routing, though that particular issue is less relevant today with modern routers which are many times more powerful than the routers for which MPLS was initially developed.
Almost immediately, it was realised that several other applications of MPLS technology could be developed, the most popular of which became Virtual Private Network (VPN) services, both in the Layer 2 and Layer 3 varieties. Layer 2 services are directly comparable to leased lines in that is it up to the consuming organisation to provide routing services over the Layer 2 MPLS connections. MPLS Layer 3 VPN services opened up a new paradigm where the service provider now manages the WAN routing for your organisation. Your routers peer with the provider, rather than directly with each other through the provider. The primary motivation behind this network design is to allow “any to any” connectivity between your locations, rather than being forced into the more traditional hub-and-spoke model, though that topology is still available with MPLS L3VPN services.
MPLS compared to Internet VPN
Organisations felt comfortable with the privacy and service levels associated with leased lines. After all, the entire circuit was dedicated to the customer end-to-end. When MPLS came along, there was an initial mistrust of the technology since it was implemented as a shared resource within the carrier network. Though privacy was guaranteed, people were concerned about oversubscription and not getting all of the bandwidth they were paying for. Service Level Agreements (SLAs) became a cornerstone of contracts with service providers to ensure customers had a path of remediation if the MPLS circuits were not performing adequately. Over time, MPLS has proven itself to be both as private and as performant as older leased-line technologies.
Remote site interconnections
Using pubic Internet-based VPNs to interconnect your remote offices has been viewed traditionally as both less secure and less reliable than private MPLS circuits. With MPLS, your connection remains within one or more private carrier networks where both privacy and redundancy are expected. Many organisations do not bother encrypting their traffic when using MPLS links. Performance levels are typically guaranteed with an SLA, which adds to their cost.
With the public Internet, traffic encryption is required to ensure end-to-end privacy since your traffic is no longer private, like with an MPLS L3VPN service. The transport of traffic across the Internet is also considered “best effort” in that there is no coordination between each of the various routers your traffic goes through from source to destination, unlike with a pre-defined MPLS LSP. You can purchase faster Internet access, but that doesn’t guarantee your traffic will reach the destination within the performance parameters you may require. Commodity Internet circuits are cheaper because they do not include guarantees like the private MPLS circuits do.
When SD WAN platforms use the public Internet for transport, generally over less-expensive circuits like broadband cable or DSL, and LTE, the proprietary “special sauce” attempts to overcome some of the limitations. For example, Forward Error Correction (FEC) is typically used to overcome packet loss. With FEC, multiple copies of the same packet are transmitted automatically, and then the SD WAN device on the other side of the WAN uses a non-corrupt packet to complete the transmission. The tradeoff is using extra bandwidth to compensate for a loss of quality in the underlying transport.
Data Centre and cloud interconnectivity
Another primary application of MPLS VPN services is to interconnect Data Centres together in a reliable, high-speed manner. Data Centres usually house the organisation’s most important systems and data, and have much higher requirements than the typical branch office with regard to network quality and availability. A fairly typical requirement of datacenter interconnectivity is guarantees of acceptable latency. MPLS circuits still excel in this regard, due both to staying within the carrier’s private network and being backed by an SLA.
Interconnectivity with cloud services is also becoming increasingly important. Applications delivered via cloud services offer the promise of accessibility from anywhere. However, just like an organisation’s custom internal applications may have strict performance requirements, cloud applications have the same need for reliable, high-speed interconnectivity. With a single vendor’s cloud service, it is easy to achieve this type of network architecture, since you simply buy the resources and capacity that you need within the cloud provider’s network. Connecting your private network to cloud services can be a different story, though. Establishing a private MPLS connection to your cloud provider from your datacenter may still produce the best level of performance, depending on where your datacenter is located compared to the cloud provider region.
How SD WAN can tie everything together
Private WAN circuits using MPLS technology can be very expensive. Any technology that promises to ease that expense is worth considering, which is why SD WAN is becoming so popular. Several years ago, the privacy offered by MPLS was important because hardware appliances capable of encrypting traffic at high speeds were still very expensive. If you used MPLS, you didn’t necessarily need to spend extra money to have your traffic encrypted. Now, the general-purpose CPUs present in most SD WAN appliances are fast enough to encrypt traffic at a rate that is acceptable for most situations, which allows the public Internet to be used for transport.
SD WAN can serve as a cost-savings method of replacing more expensive MPLS circuits for many different situations. However, like most technologies, it is not appropriate for all situations. Like the expensive appliances previously required to encrypt high volumes of traffic, this is still true with SD WAN, since most appliances are using general-purpose CPUs which have difficulty encrypting multiple gigabits per second of traffic. This particular constraint will also be alleviated over time as technology improves. Until then, private MPLS circuits are still necessary when moving very high volumes of traffic over the WAN, such as tens or hundreds of gigabits per second.
Privacy and encryption aside, Internet-based transport could not be considered a serious replacement for private connections until fairly recently due to the underwhelming reliability of the access technologies, such as broadband. Over time, this has also improved dramatically as enhancements to the underlying technologies have been developed, such as the progression of the DOCSIS cable standard along with newer forms of DSL and cellular wireless technologies. Despite the advancements in reliability, commodity Internet circuits are still “best-effort” and not backed by an SLA.
SD WAN attempts to overcome these limitations with techniques like FEC, but if your application requires this high level of performance, you may still be served best with a private MPLS service. A service provider might not always strictly meet their SLA, but you will at least have a remediation process available to you. Since MPLS services remain private to a carrier (or a few carriers within inter-carrier agreements), you will usually have less latency across an MPLS circuit than with a general Internet connection.
SD WAN providers promote Internet-only transport due to the potential cost savings, but the good news is that SD WAN can also work with MPLS circuits as well for when you need the performance and latency guarantees associated with MPLS. SD WAN is transport-agnostic. It doesn’t care what is in the middle, as long as it can reach the other side.
When using SD WAN with MPLS, you have the option of having the SD WAN appliance make the forwarding decision. Based on policy and current link conditions, forwarding may happen over the Internet link, or it may happen over the MPLS link. This doesn’t necessarily eliminate the encryption performance problem, where the SD WAN appliance must encrypt all traffic with its general-purpose CPU, but the MPLS circuit may be more performant from a latency perspective.
Where will MPLS fit in the future?
MPLS as a forwarding paradigm is here to stay for the foreseeable future. While the volume of MPLS circuits may go down from a customer perspective, MPLS will continue to be used in carrier backbones for years to come, especially as newer enhancements such as Segment Routing are developed and deployed. Likewise, customer MPLS circuits are still required for traditional high-speed datacenter interconnects (DCI) as SD WAN appliances cannot yet process tens and hundreds of gigabits per second of traffic.
Private MPLS connections are also currently important when connecting on-premises datacenter applications to custom public cloud-hosted services. This, too, will rely less on dedicated private connections over time as SD WAN vendors create virtualised appliances that can run in the same public cloud network and act as another endpoint in your WAN.
Can you completely remove MPLS circuits from your environment and replace it with public Internet-based SD WAN? Depending on your organisational requirements, this might be a possibility. Ultimately, you must carefully evaluate your current and predicted future needs to determine what is right for your organisation. Migrating to SD WAN does not have to be an all-or-nothing transition. Keeping private MPLS links may be appropriate for some locations in your organisation, while removing them to save costs in favour of Internet transport may be feasible in other locations.
Private MPLS circuits will be necessary in some capacity for most organisations for the foreseeable future, though this may change at some point with the development of new technologies. The expense of using private MPLS circuits may still be worth it if you risk losing significant revenue due to potential poor-quality best-effort public Internet links with performance so low that even the proprietary technologies of SD WAN are unable to remediate the connection. This situation is becoming increasingly rare, though, and will only improve with time.
Despite what vendors and industry pundits may claim, no technology is inherently good or bad. All technologies are developed to solve specific issues. Those issues may grow and change over time, and this is why it is critical to continually evaluate your organisation’s specific needs and requirements against what technologies are available. Your organisation may have application architectures that support moving away from MPLS to a pure public Internet-based SD-WAN. Many organisations will still have very tight requirements that necessitate the guarantees inherent in MPLS transport. Regardless, MPLS is going to run the core of carrier networks for quite some time into the future.
About Jedadiah Casey
Senior Network Engineer for 5 years General IT/sysadmin experience 10 years prior Bachelor of Science degree in Information Systems Certifications: Cisco CCNP Routing & Switching, CCDP Network Design, CCNA Routing & Switching, CCNA Wireless, CCNA Industrial, CCNA Service Provider Certified Wireless Network Professional CWNA VMware VCP-DCV Juniper JNCIA Working toward Cisco CCIE R&S, first lab attempt was June 2018.