& cplSiteName &

Magic Quadrant or Gartner 'Graft'?

Ray Le Maistre
11/12/2015
67%
33%

Mention Gartner's Magic Quadrant to anyone in the communications networking industry and you'll get a "Can't live with it, can't live without it" look: Everyone knows it's influential and affects market perceptions and initial selections for RFI (request for information) processes, but many on the supplier side will say how much they dislike or even despise it -- mostly off the record.

Some companies, though, feel strongly enough to go public with their criticisms of the influential process, which ranks technology suppliers based on their "ability to execute" and "completeness of vision" and places them in one of four quadrants -- Leaders, Challengers, Visionaries or Niche Players.

"Leaders," of course, is the quadrant every company wants to be in.

NetScout Systems Inc. (Nasdaq: NTCT), for example, was so angered by its treatment in the Magic Quadrant process related to Gartner's March 2014 report on the Network Performance Monitoring and Diagnostics market that it initiated legal action against the research and consulting company: That legal case is ongoing and we'll come back to that later. (See NetScout Sues 'Pay-to-Play' Gartner.)

Now AsiaInfo Inc. (Nasdaq: ASIA), a privately held BSS vendor with estimated revenues of more than $600 million per year generated mostly from China's three main operators, has decided to go public to voice its exasperation and frustration with the Gartner Inc. process. (See AsiaInfo-Linkage Goes Private.)

AsiaInfo's disgruntlement is related to Gartner's Magic Quadrant for Integrated Revenue and Customer Management (IRCM) for CSPs, the latest version of which was published in October.

Andy Tiller, Director of International Marketing Communications at AsiaInfo, tells Light Reading that it was frustrated with the process for the 2014 IRCM report, and complained at the time to Gartner's ombudsman. Now it feels very aggrieved at the outcome of the 2015 report.

Relative positions
Tiller notes that AsiaInfo was identified as a Challenger in 2014, alongside CSG, Huawei, Redknee and ZTE, while the giants of the sector -- Amdocs, Ericsson, NetCracker and Oracle -- were positioned as Leaders. Despite not being happy with Gartner's processes -- "the Magic Quadrant analysts don't accept our offers of briefings, they don't talk to our customers in China and the questionnaire and anecdote processes used by the analysts are just not rigorous" complains Tiller -- AsiaInfo didn't have any problems with its position in 2014 relative to its rivals, which, for the deals it is chasing, tend to be Huawei and ZTE rather than those in the Leaders quadrant.

But Tiller was shocked when he saw this year's report: Despite new wins in Asia and a breakthrough in Europe, where AsiaInfo now has 300 staff and is engaged in commercial deployments with Telenor, its position in the IRCM quadrant was largely unchanged. (See AIS Thailand Deploys AsiaInfo's BSS and AsiaInfo Lands New Euro Deal.)

That in itself isn't Tiller's problem: What he's fired up about is that other companies, ZTE in particular, have made the leap into the Leaders quadrant without, in AsiaInfo's opinion, having made any great headway in the sector.

Huawei has also made the transition to the Leaders category but Tiller conceded that it at least has made significant strides in the IRCM market and is engaged in a number of major BSS transition programs. (See Telefónica Bets on IT Transformation and Orange Looks to Huawei for IT Overhaul.)

"It's not so much our position that we're angry about -- it's our position in relation to others," says Tiller. "The big shock is that ZTE has shifted from being a Niche Player to a Leader in two big gains but we see no evidence in the market as to why that should be. It may have improved but, if so, then others, including us, have improved also. But our position in the quadrant is exactly where we were last year, even though we now have a live European deployment and a major customer in India. They told us last year that we had to find some valid [international] customers and prove ourselves and we have done that, but we still haven't moved." (See AsiaInfo Forms Joint Venture With Reliance .)

Tiller continues: "Our biggest annoyance is with the position of ZTE," which is active in the market via its ZTEsoft unit. (See ZTE Shows Off Its Soft Side.)

"ZTE's position as a Leader shows us that this is a not a level playing field and that the quadrant is not accurate or rigorous. It's very annoying. It doesn't impact us at the end of the procurement process -- it impacts us at the beginning, when operators decide which companies to contact for RFIs," notes the AsiaInfo man.

"If you are a major client, you get time with the Gartner analysts. But we are a very small client. The ombudsman told us we should increase our efforts to engage with the analysts but those analysts then declined our offer of a briefing at the TM Forum Live event in Nice this year [the major annual industry event for the revenue and customer management market]. They agreed to meet in the end, but only showed up for ten minutes. We wouldn't say that the process is corrupt -- I don't think you can just buy your way into the quadrant -- but we would say the process is very flawed. The way they collect the information is flawed -- our biggest customers are in China but the Gartner analysts don't engage with them in their language to collect information and anecdotes … Gartner denied a request from us to have their questionnaire translated into Chinese. And even though there is a review process of the information in the report before it's published, there are still factual inaccuracies published."

Tiller wasn't finished: "We will complain again to the ombudsman but we don't expect that to have any impact. We are fed up with the process and have decided to go public with our grievances. We wouldn't say it's directly pay-to-play, it's just that the analysts spend a lot of time with the companies that pay them a lot of money. If I thought this was a rigorous process then that would be fair enough -- but it isn't."

Gartner was approached by Light Reading for comment on AsiaInfo's claims -- it stated:

    Vendors communicate information about their products and services to research analysts through meetings with analysts known as 'vendor briefings.' Vendors that request vendor briefings typically are seeking coverage by research analysts of their specific products and services. Gartner manages the vendor briefing process to match specific technology companies with the most appropriate research analysts, as well as to ensure that a specific vendor's scale and scope are appropriate for Gartner research studies. Vendors may obtain access to research analysts through vendor briefings, regardless of whether or not they are Gartner clients.

NetScout perseveres with legal action
Tiller's views largely mirror those of NetScout, but while the AsiaInfo man shies away from accusing Gartner of "pay-to-play" tactics, NetScout made that accusation a key part of its legal action against the consultancy.

So what is the status of that process? Well, after more than a year, it is still in motion. NetScout, which had to revise its complaint against Gartner earlier this year to eliminate irrelevant details and inapplicable references, will say only that "both sides are in the middle of discovery still and a court decision on one of the motions filed could be made this year."

In the meantime it continues to criticize Gartner's methods, describing the Magic Quadrant for Network Performance Monitoring and Diagnostics (NPMD) Report, 2015, as "flawed" in this press release issued earlier this year.

With respect to NetScout, Gartner states in an emailed response to questions from Light Reading: "The complaint filed by NetScout Systems, Inc. stems from its unhappiness with NetScout's placement in a Gartner Magic Quadrant. While it's not our practice to discuss pending litigation, we do consider this complaint to be without merit and intend to defend ourselves and the integrity of our research processes vigorously. We remain committed to providing our clients with independent research and advice about the products and services that we cover and upon which they have relied for decades."

— Ray Le Maistre, Circle me on Google+ Follow me on TwitterVisit my LinkedIn profile, Editor-in-Chief, Light Reading

(5)  | 
Comment  | 
Print  | 
Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
kq4ym
50%
50%
kq4ym,
User Rank: Light Sabre
11/25/2015 | 5:55:15 PM
Re: Magick
I would guess the lawsuit is not going to be too successful, although the pressure put on Gartner may well move them to revise or reconsider their methodology for smaller outfits. It's pretty unlikely anyone is going to find a pay to play program going on I would think. It may have some truth that it is "not a level playing field and that the quadrant is not accurate or rigorous," but proving that and paying attorneys to find the magic bullet is not easy.
tpoulos
0%
100%
tpoulos,
User Rank: Lightning
11/12/2015 | 3:53:35 PM
So thankful
Well done Ray for having the balls to air AsiaInfo's concerns. We at DisruptiveViews have been the lone voice in the wilderness for years raising these concerns. So many have agreed but we're too frightened to speak out publicly for fear of retribution. It's about time senior people stopped relying on 'magic' and started making business decisions based on fact, maybe even logic. While they mull over Gartner's squares the rest of the digital world is going on gut feel, exciting new vendors and innovative partners instead of staying bogged down in historic gobbledygook and marketing budget power plays.
mendyk
50%
50%
mendyk,
User Rank: Light Sabre
11/12/2015 | 2:14:14 PM
Re: Magick
Yes, I've heard lots of off-the-record grumblings over the years, from IT suppliers as well as networking vendors. But it's a staggering leap from sotto voce whispers to going public with complaints and suspicions. Lawyers almost certainly will be involved.
Ray@LR
50%
50%
[email protected],
User Rank: Blogger
11/12/2015 | 2:00:08 PM
Re: Magick
But still much reluctance to go public with views.

Alrady I have received emails from execs saying they have the same experienxc as AsiaInfo but they don't want to share those opinions... the quadrant does indeed cast a spell. 
mendyk
50%
50%
mendyk,
User Rank: Light Sabre
11/12/2015 | 11:49:53 AM
Magick
Questions and speculation about Magic Quadrant have circulated for over a decade, but MQ remains a strong and influential tool -- or maybe it's a cudgel -- in the technology supplier evaluation process. Those on the outside looking in will continue to wonder about the selection criteria. Others of us will continue to wonder why MQ continues to hold sway with technology buyers. Laziness and perceived safety certainly play a role. Those of a certain age may remember the axiom, "Nobody ever got fired for choosing IBM."
Featured Video
Flash Poll
Upcoming Live Events
March 12-14, 2019, Denver, Colorado
April 2, 2019, New York, New York
April 8, 2019, Las Vegas, Nevada
May 6, 2019, Denver, Colorado
May 6-8, 2019, Denver, Colorado
September 17-19, 2019, Dallas, Texas
October 1, 2019, New Orleans, Louisiana
October 10, 2019, New York, New York
November 5, 2019, London, England
December 3, 2019, New York, New York
December 3-5, 2019, Vienna, Austria
All Upcoming Live Events