Dish Network officially began selling its 5G network earlier this month, and some early tests show dramatic performance inconsistencies.

"The phone often had trouble making connections on the Dish network in Ithaca [New York], sometimes stalling and dropping back to AT&T, but I got some web page loads and speed tests done," wrote PCMag's Sascha Segan of his Dish tests across three cities in the state. "Speeds were incredibly slow at 1-5 Mbit/s down, and less than 1 Mbit/s up."

The analysts at Signals Research Group (SRG) came to similar conclusions of their Dish tests in Las Vegas.

"To summarize our findings, the overall network performance wasn't terrible, and there were some indications of quite good performance, but there was no consistency in the performance and consistency is something that consumers expect," they wrote in a new report.

Figure 1: (Source: Dish)

Both Segan and SRG found that Dish's 5G network routinely pushed customers onto AT&T's network when it wasn't able to provide a reliable connection, even deep inside markets where Dish is advertising its coverage. Segan noted that Dish's Samsung Galaxy phone appeared to rely on two SIM cards, one for Dish's network and another for AT&T's.

In-marking roaming

Broadly, the findings indicate the importance of in-market roaming agreements for Dish as it builds its 5G network across the country. Such services allow the company to push its customers onto another network when it can't get them connected.

Dish inked a $5 billion roaming agreement with AT&T last year that gives Dish the option to use AT&T's network "even within the markets where Dish is deploying its own 5G network." That agreement appears to play a major role in Dish's new Project Genesis offering.

Further, Dish just this week announced a new $3.3 billion roaming deal with T-Mobile that also includes "in-market roaming to Dish's wireless customers within the T-Mobile network footprint (which includes areas within the markets where Dish is deploying its own 5G network)."

Concluded SRG of the situation: "AT&T may like the extra revenue, but it can't be good for the Dish Wireless bottom line."

VoNR

Perhaps Dish's most glaring inconsistency involved its implementation of 5G Voice over New Radio (VoNR). Just like Voice over LTE (VoLTE) on 4G, the technology promises to push customers' voice calls onto a 5G network. But according to SRG's findings, Dish's VoNR service proved to be extremely unreliable in Vegas.

"We also observed VoNR voice quality that was as good, if not better, than anything we've captured with VoLTE, but there were also frequent instances with very poor voice quality, even while stationary and with good RF conditions," the analysts wrote.

In a post on LinkedIn, SRG's Emil Olbrich highlighted Dish VoNR testing results where a first call sounded crystal clear and a second call, placed in virtually the exact same conditions, was completely garbled.

"I see this as a silver lining," he said, explaining that the Dish 5G network appears capable of providing excellent quality and speedy connections. However, he acknowledged that "the consistency, they're going to have to fix that."

Dish's efforts to launch VoNR are noteworthy considering T-Mobile is also rolling out the technology, albeit in just two cities and on one phone. AT&T and Verizon, meanwhile, haven't launched that technology nor the standalone version of 5G required to support it.

Speeds and feeds

In terms of upload and download speeds, Dish's 5G network generally trailed those of its competitors, as some analysts warned it would.

"Dish has a big problem in Upstate New York right now, and it's the number 5," Segan wrote. "The problem isn't the G, it's the MHz. Everywhere I went, the Dish network was using only 5MHz of band n71, alone. That's '4G network in 2012' levels of spectrum usage, and yep, it has about the performance of a 4G network in 2012."

The SRG analysts wrote that Dish's network in some cases outperformed Rakuten's network in terms of download performance – which is noteworthy considering the Japanese operator is the only other provider in the world to build a widespread 5G network using open RAN principles. But the analysts found that Dish's uplink network performance was "unsatisfactory" and substantially below what they found on T-Mobile's 5G network in the city.

A final item, noted by Segan, is that Dish so far is only using a small portion of its overall spectrum holdings. That's partly because it's still waiting on equipment for some bands. But he also speculated that Dish's launch is really just an "extended beta" that will allow the company to continue to refine its network over time.

The open RAN component

Structurally, Dish's network is much, much different from the networks operated by AT&T, T-Mobile and others. It's designed to work entirely in the cloud, and it promises to use open RAN specifications to connect products from a wide range of vendors.

However, based on the early tests of Dish's network, that setup clearly still needs work. Moreover, Dish had initially hoped to launch its network in multiple cities last year – which reflects the difficulties the company is having in getting its systems up and running.

"We're not sure where the fault lies," wrote the SRG analysts. They said they were unable to sufficiently test Dish's open RAN networking components in order to isolate specific vendors.

They continued: "Not knowing the root cause of these issues makes it difficult to theorize on the implications for the future success of open RAN. A 'simple' software upgrade could make all the problems go away while we can't fault the vendors if the cell density and network design weren't on par with other operator networks. Another shortcoming we observed was the one-way latency in both the downlink and uplink directions. This could be an RF issue, a vendor scheduling shortcoming, or something to do with AWS (Amazon Web Services). Hopefully we'll get the chance (and we will) to do more extensive testing and data analysis in the network in the coming months. Given what we know and what we suspect, we believe there are equal opportunities for all parties to improve the network performance."

Related posts:

— Mike Dano, Editorial Director, 5G & Mobile Strategies, Light Reading | @mikeddano