& cplSiteName &

Mellanox, Intel & the Data Center Bottleneck

Brian Santo

Intel Corp. (Nasdaq: INTC) and Mellanox Technologies Ltd. (Nasdaq: MLNX) are both making gradual progress in their race to unclog a bottleneck that arises when scaling computer networks up to massive numbers of nodes or processors (CPUs). The problem is a present concern in high-performance computing (HPC), and if left unresolved it threatens to hobble data centers, and not just because they're beginning to rely on HPC for big data analytics.

The basic problem is that the speed gain in each successive generation of processor is increasingly more modest. This phenomenon was an important factor in the development of the multicore approach; if you can't get faster, you can do more by using more processors simultaneously. But the increase in processing power that results does not scale linearly with the number of processors. This approach, too, is delivering diminishing returns.

Supercomputers and data centers are different beasts, but what they have in common is a dependence on massive numbers of processors, and a perpetual need for more and faster processing.

But with faster and more both running out of steam, what's the solution? Intel and Mellanox both propose making interconnect more efficient, but disagree on how. Intel proposes its Omni-Path Architecture (OPA). Mellanox, meanwhile, is pegging its future to intelligent interconnect. (See Mellanox Capitalizes on 25G Transition.)

The bottleneck is subtext for some of the activity in other corners of the server chip market. It is said to be one of the reasons why Google (Nasdaq: GOOG) is looking at alternatives to Intel processors for its data centers. Three years ago, Google entertained the possibility of designing its own data center chips, though it never commercialized anything. Earlier this year, rumors were widespread that Google was looking at Qualcomm's server chips, which to this day Qualcomm Inc. (Nasdaq: QCOM) is showing only to the largest potential customers. Qualcomm is one of the companies looking to compete with Intel with ARM-based server chips. Cavium Inc. (Nasdaq: CAVM) is another. (See Cavium Debuts SoC for Data Center Servers.)

Intel's OPA is an alternative to Infiniband interconnect that takes a fabric approach to scaling up nodes. The new architecture encompasses PCIe adapters, silicon, switches, cables, and management software. OPA boasts latencies roughly on par with EDR Infiniband, Intel documents say, but has other improvements to overall network efficiency, such as link-level error correction and traffic optimization that gives preference to higher-priority traffic.

Intel announced the architecture last year, and earlier this summer launched yet another version of its Xeon processor, the Knights Landing Xeon Phi, which is optimized in ways that make it useful to support OPA.

Intel made the Knights Landing Xeon Phi available in India for the first time today (Sept. 26), hoping to attract HPC business in that country. Intel previously announced that several supercomputing centers (e.g., the Texas Advanced Computing Center, the Pittsburgh Supercomputer Center) are using the architecture. More recently, Intel and Fujitsu announced that the latter is building a supercomputer based on Knights Landing and OPA for the Joint Center for Advanced High Performance Computing (JCAHPC) in Japan; the date of completion for that project is this December.

Mellanox is proposing to make processors more efficient by relieving them of certain housekeeping tasks and -- just as importantly if not more so -- doing some of the calculations away from CPUs.

That of course plays to the company's strengths as a leading supplier of data center interconnect, but Mellanox Vice President of Marketing Gilad Shainer insists Mellanox's approach is the only way to truly solve the bottleneck.

In data centers, almost every server -- itself an aggregation of multicore CPUs -- is perpetually fetching data from multiple others and constantly aggregating and analyzing it.

What Mellanox proposes is to analyze data wherever it is, as the data moves, by putting some processing capability in interconnect, in the switch network and in NICs. It would have to run on the switch ASIC itself, rather than adding a CPU, because that's just moving the problem, not solving it.

Want to know more about communications ICs? Check out our comms chips channel
here on Light Reading.

The first step that Mellanox is taking, Shainer told Light Reading, is moving algorithms for data aggregation and data reduction off the CPUs.

"Now you can do things much faster because you don't need to wait for data. Also, you minimize the amount of data that has to move," Shainer said.

He said you can accelerate those specific algorithms, getting a 10X improvement in those specific algorithms.

"You will see a growing number of data algorithms moving away from the CPU. The network will not become a general purpose CPU, because that's what you have a CPU for. But you can move some of these algorithms," he said.

Another avenue to pursue is to start using in-network memory. "You start having pieces of information so that it's available to all processors," Shainer noted. Commonly used data can be stored in multiple places (perhaps in NICs) so that it is more readily available.

Mellanox's approach is not necessarily inimical to Intel's. It can be accomplished without the participation of CPU vendors, though it would work much better if CPU vendors were to provide certain interfaces that would make the process work smoother.

— Brian Santo, Senior Editor, Components, T&M, Light Reading

(1)  | 
Comment  | 
Print  | 
Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
User Rank: Light Beer
10/3/2016 | 3:25:30 AM
Data Center Interconnect
Trends and Directions in Data Center Interconnect has been changing and this article was certainly helpful to me . Thanks for this informative article .
From The Founder
Following a recent board meeting, the New IP Agency (NIA) has a new strategy to help accelerate the adoption of NFV capabilities, explains the Agency's Founder and Secretary, Steve Saunders.
Flash Poll
Live Streaming Video
Charting the CSP's Future
Six different communications service providers join to debate their visions of the future CSP, following a landmark presentation from AT&T on its massive virtualization efforts and a look back on where the telecom industry has been and where it's going from two industry veterans.
LRTV Interviews
BT's McRae Sheds Light on 4K Strategy

5|25|17   |   4:45   |   (0) comments

At Light Reading's Big Communications Event 2017 in Austin, Texas, BT Group's Chief Network Architect Neil McRae talks about what it took for BT to broadcast live sports in 4K. Catch up with all our BCE coverage at http://www.lightreading.com/bce.asp.
From the Founder
How the NIA Aims to Advance NFV

5|25|17   |   08:07   |   (0) comments

Following a recent board meeting, the New IP Agency (NIA) has a new strategy to help accelerate the adoption of NFV capabilities, explains the Agency's Founder and Secretary, Steve Saunders.
LRTV Custom TV
Better Solutions That Address Growing Scale

5|25|17   |     |   (0) comments

For Comcast, the X1 rollout and 17-fold increases in broadband speeds in the past 16 years are among factors driving the need for Energy 2020 solutions that reduce cost and consumption, says Mark Hess.
LRTV Custom TV
Ethernity Network Delivers Instant Offloading of Network Functions With All-Programmable Intelligent NIC

5|25|17   |     |   (0) comments

David Levi, CEO of Ethernity Networks, explains that programmability of the hardware makes the company's All-Programmable Intelligent NIC uniquely beneficial for communications service providers that need advanced data appliances with agile support of virtualization. Utilizing the company's patented network processing technology, Ethernity offers data path ...
LRTV Documentaries
BCE 2017: Vodafone Gets Obsessed With Cloud-Native

5|25|17   |     |   (0) comments

Vodafone's Matt Beal updates us on Project Ocean and explains why simple virtualization isn't enough of a goal for network transformation. Catch up with other BCE 2017 keynotes and news at http://www.lightreading.com/bce.asp.
LRTV Documentaries
BCE 2017: Intel's Take on Network Transformation

5|24|17   |     |   (0) comments

In this BCE 2017 keynote, Lynn Comp discusses Intel's vision for areas such as analytics, automation and service assurance. For more videos and BCE coverage, see http://www.lightreading.com/bce.asp.
LRTV Documentaries
Order From Chaos: The Steve Saunders BCE Keynote

5|24|17   |   17:27   |   (0) comments

Kicking off BCE 2017, Light Reading founder Steve Saunders lays blame for NFV's slow ramp-up and urges telecom to return to old-fashioned standards building and interoperability testing.
Think of this as the video sequel to the recent columns he's written about NFV and the prospect of a telecom app store. (See

LRTV Documentaries
Service Provider Panel: Partnering in the Digital Era

5|22|17   |     |   (0) comments

Coopetition has always been part of telecom, but the ecosphere now includes data centers, vendors, apps developers, cloud service providers and Internet content providers. This BCE 2017 panel explores the new attitudes among network operators as to the value and variety of ...
LRTV Interviews
Site Demo: AT&T's IoT Flow Platform

5|19|17   |   04:25   |   (0) comments

At AT&T's R&D center in Tel Aviv, Israel, project leader Eyal Segev talks about the operator's Flow platform and how it helps to prototype IoT applications.
LRTV Documentaries
Agent of Change: A Q&A With AT&T's John Donovan

5|18|17   |     |   (0) comments

Carol Wilson talks with the man leading AT&T's transformation efforts about the challenge of change.
LRTV Documentaries
BCE Service Provider Panel: The New Business Realities

5|18|17   |     |   (0) comments

For virtualization to happen, the telecom industry first has to grapple with key functional aspects of SDN and NFV that need to be universal, such as onboarding of virtualized network functions and federation of software-defined networks.
LRTV Interviews
BCE Service Provider Keynote: CenturyLink

5|16|17   |   22:32   |   (0) comments

Aamir Hussain leads the Product Development and Technology organization at CenturyLink, which includes the company's information technology function. He is an experienced senior technology executive with more than 25 years of proven success in the implementation of global technology operations, operationalization of complex technology, infrastructures and business ...
With the mobile ecosystem becoming increasingly vulnerable to security threats, AdaptiveMobile has laid out some of the key considerations for the wireless community.
Hot Topics
Cities Clamor for More Clout at FCC
Mari Silbey, Senior Editor, Cable/Video, 5/23/2017
What's Blocking 4K TV Today
Alan Breznick, Cable/Video Practice Leader, Light Reading, 5/22/2017
Sonus & Genband Finally Combine to Form $745M Company
Dan Jones, Mobile Editor, 5/23/2017
Fright Wigs & Cocktails: BCE 2017 in Pics
Mitch Wagner, Editor, Enterprise Cloud, 5/19/2017
Apple Looking to Cook 5G Test Devices
Dan Jones, Mobile Editor, 5/24/2017
Like Us on Facebook
Twitter Feed
BETWEEN THE CEOs - Executive Interviews
One of the nice bits of my job (other than the teeny tiny salary, obviously) is that I get to pick and choose who I interview for this slot on the Light Reading home ...
TEOCO Founder and CEO Atul Jain talks to Light Reading Founder and CEO Steve Saunders about the challenges around cost control and service monetization in the mobile and IoT sectors.
Animals with Phones
What Brogrammers Look Like to the Rest of Us Click Here
Live Digital Audio

Playing it safe can only get you so far. Sometimes the biggest bets have the biggest payouts, and that is true in your career as well. For this radio show, Caroline Chan, general manager of the 5G Infrastructure Division of the Network Platform Group at Intel, will share her own personal story of how she successfully took big bets to build a successful career, as well as offer advice on how you can do the same. We’ll cover everything from how to overcome fear and manage risk, how to be prepared for where technology is going in the future and how to structure your career in a way to ensure you keep progressing. Chan, a seasoned telecom veteran and effective risk taker herself, will also leave plenty of time to answer all your questions live on the air.