In my last post, we looked at the various networking-specific factors which affect application performance on the WAN. While an exhaustive review of all the possibilities would take far more than a single blog entry, let’s take a look now at the techniques vendors have implemented over the last 10 - 15 years to address these factors.
Last time, we saw that latency, packet loss and bandwidth are the 3 key factors which drive application performance. Latency has a relatively fixed component based primarily on the speed of light for the distance a packet must travel on the one hand, and a variable component from the queuing congestion experienced at packet forwarding devices like routers on the other. Packet loss is most of the time a function of queuing congestion, and partly a result of bit errors on unreliable links. Bandwidth, of course, is typically limited on the WAN simply because it has a monthly charge, and it’s expensive.
Three other major factors affecting WAN application performance, which themselves are hugely impacted by latency and packet loss, are the bandwidth-delay product, TCP’s congestion avoidance mechanisms, including additive-increase-multiplicative-decrease (AIMD) scheme, and the “chattiness” of certain applications or protocols, especially Microsoft’s CIFS protocol and HTTP.
We know that for best WAN performance, we want to avoid packet loss and high latency as much as possible. Successful techniques will either address bandwidth limitations or application chattiness, or will hide loss or high latency from the application, or will avoid the loss or high latency (i.e. jitter) occurring in the first place. The best solutions, of course, will address multiple of these.
Link Aggregation – e.g. MultiLink PPP – has been around for a number of years. Bonding multiple identical WAN connections together, essentially at layer 1, link aggregation allows greater overall bandwidth by combining multiple similar circuits. Examples include bonded ISDN, and bonded T1 or E1 services. [In the LAN, bonded Ethernet works similarly]. What these solutions have in common is that each of the links is identical, and has extremely low loss and extremely low jitter. In such cases, simple layer 1 bonding techniques and MLPPP can work very well to deliver greater bandwidth. This usually comes, of course, at a near-proportional increase in WAN service cost.
Bandwidth management, also sometimes loosely referred to as “QoS” (Quality of Service), is a way to deal with limited bandwidth to ensure that loss-sensitive or latency-sensitive applications like VoIP, videoconferencing or Citrix get “favored” use of bandwidth, essentially prioritizing certain traffic over others. QoS/bandwidth management has been available in data networks “forever”. This essential technique for running real-time applications like VoIP can be sufficient if a single provider owns the WAN infrastructure, and can often improve performance for interactive applications like web pages when sharing links simultaneously with file transfers.
Bandwidth management / prioritization usually refers only to the outbound direction of packet forwarding. All WAN forwarding devices – routers, WAN Optimization products, VPN products, Adaptive Private Networking appliances, etc. – available today do this type of bandwidth management and prioritization.
In the late 1990s, Packeteer (now part of Blue Coat) pioneered a technique known as TCP rate control to improve the use of inbound connections, especially public Internet connections. By messing with the TCP window size and when it released TCP acknowledgement packets for a given TCP flow, Packeteer was able to reduce congestion and enable better performance of real-time and interactive applications coming into a WAN link.
Desktop virtualization – think Citrix remote desktop product, now known as XenDesktop – solves the problem a different way. By running the entire application remotely, performance issues related to bandwidth and the chattiness of the underlying/original application are rendered moot, as the application now runs entirely on a LAN – or sometimes entirely in a virtual machine – at a data center. Only the GUI, mouse clicks, key clicks, and screen results are transmitted across the network. Note, however, that desktop virtualization is both a solution to WAN app performance, as well as an “application” itself for which performance over the WAN – especially lossy or very high latency WANs – must be solved/addressed.
TCP termination in a networking device solves the bandwidth-delay product issue. Specifically, TCP termination solves the problem where TCP transfers are limited by the 32KB – 64KB maximum TCP window sizes on typical Windows or Linux OS hosts, which impede the ability to use large WAN links for transfers over long distances. This obviously solves problems with high latency. Less obviously, it also improves performance in the face of small amounts of loss, since TCP’s AIDM algorithm reduces the TCP window size by half upon a single lost packet. Under small amounts of loss, the window size of a terminated flow will typically be much larger than the window size of the unterminated flow, and so performance is maintained at a higher rate using TCP termination in this case. [Note that under high packet loss, this benefit is muted, as the window size of the flow using TCP termination is unlikely to be much or even at all larger than the window size for an unterminated flow.]
Caching, e.g. as Akamai is well known for for public Internet applications, can solve the speed of light issues involved in getting objects/data across long distances, and can sometimes help a bit with bandwidth limitation issues, depending on whether the cache is at the local site versus whether it is (as for Akamai) in the network cloud but at a point very close in terms of latency to the client. Such caching historically is an application-specific solution.
WAN Optimization products from companies like Peribit (acquired by Juniper) and Riverbed Technology came along early in the 2000s to focus on the issues of WAN application performance. They first attacked the limited bandwidth available on WANs, freeing up bandwidth by doing data compression. Initially, the compression was completely memory-based, then Riverbed introduced disk-based compression/caching, improving compression ratios substantially in some cases. Compression has been around forever, but where in the ’80s and ’90s it might have yielded an average of 40% bandwidth improvement – and at a prohibitive cost in terms of CPU consumption at the time – memory-based compression delivers an average of 2x bandwidth improvement now, and disk-based compression typically delivers improvements in the range of 2x – 4x.
By combining compression with CIFS-specific optimization, WAN Optimization not only delivers a bandwidth benefit on average (with the occasional huge benefit when the same file is transferred then 2nd, 3rd, 4th, etc. time), but also solves CIFS’ notorious chattiness problem.
WAN Optimization products also typically do HTTP-specific optimization, reducing the chattiness problems with that protocol.
Wide Area File Services (WAFS), or more generally distributed files services which do replication, such as Microsoft’s DFS (Distributed File Service) with Replication, are a specific implementation of caching/replication for files. They deliver true LAN-speed performance for files which have been distributed to remote locations and stored on a hard disk on a server device there. In the era of very thin WAN pipes that has been the case for the last decade plus, WAFS/DFS was appropriately beaten in the market by WAN Optimization products which not only solved more problems than just file access, but solved them better where downstream bandwidth is at such a premium that pre-positioning of files in a distributed fashion is not practical.
In my next post, we'll look at the innovations Talari is bringing to the table with Adaptive Private Networking technology to address application performance on the WAN.
Comments