In a recent headline, NY Attorney General Eric Schneiderman invited citizens to test their Internet speeds and submit the results online as part of an ongoing probe.
"New Yorkers should get the Internet speeds they pay for," said Schneiderman. Unfortunately, Schneiderman's current methodology may not be adequate for the task.
The main focus of Mr. Schneiderman's investigation appears to be throughput. He announced his office has created a new online broadband test on a site called Internethealthtest.org that will capture a customer's "throughput" -- or the speed at which customers actually access Internet content. Measuring from real end users is a great idea, but using throughput alone as a metric isn't enough. It's also important to understand Round Trip Time (RTT) and availability as major contributors to ISP performance.
Let's start with throughput and the three companies that Schneiderman cited. (Note: For throughput, larger numbers are better.)
In the 25th percentile (the 25th "lowest throughput" set of real users for each network), there seems to be a real advantage to Cablevision. However -- if you are lucky enough to be on the 95th percentile of "highest throughput" users -- clearly it would be better to be on Verizon!
The Importance of RTT (Round Trip Time)
Schneiderman is certainly not the first person to think that throughput is the most relevant factor. Over the years as an Internet community, we have been bombarded with advertising that states our ISP's "bandwidth" is the key to speed. Unfortunately this is not the case. It certainly helps, and, if offered a "faster" connection, we would all take it, but there are some fundamentals to how the Internet works that throughput doesn't cover.
One of the key metrics for evaluating different suppliers is Round Trip Time (RTT) -- the time it takes for a request to go to a server and the client to receive a response from it. Let's look at why this metric is very important when it comes to user performance across the web and how RTT and the TCP protocol actually affect the performance of a web asset or web assets.
The part of TCP we will look at here is TCP's Maximum Window Size. There are many other factors such as slow-start, re-transmit time out and window scaling that also affect performance, but Maximum Window Size is useful as an example.
The Maximum Window Size is defined as the maximum receivable data size (specified in bytes) that the client can buffer during a connection. It is usually established during the TCP hand-shaking process and will mean that the sender will limit its sending Maximum Window size to this value and only send more data once it has received an acknowledgement from the receiver -- i.e., an Ack.
Now why is this important? Well, if we can only send a certain amount of data in each round trip and then must wait for an acknowledgement, then the actual transmission time of that data over the wire becomes more important. Simply, the faster I get the acknowledgement that my data got to its destination, the faster I can send the next bit of data.