& cplSiteName &

Cloudbleed Lessons: What If There's No Lesson?

Curtis Franklin
3/15/2017
50%
50%

Whenever a serious computer or network security issue becomes public, one of the first questions IT professionals ask is, "What lessons can we learn?" It's a polite way of phrasing the real question: "How do I keep my company out of the news for something like this?" But what's a professional to do when the best answer to the question might well be that absolutely nothing any reasonable company might have done would stop the problem? That's a very different question.

2017 has seen a spate of news generated by errant keystrokes, from the "Cloudbleed" vulnerability that exposed millions of pieces of personally identifiable information to the AWS outage that brought large portions of the Internet to its knees. Finding a single keystroke going awry makes the classic "needle in a haystack" analogy insufficient. Finding a needle in a thousand-acre field of haystacks might be more like it -- and that's something that may simply go beyond reasonable.


Don't get left in the dark by a DDoS attack -- learn best practices to strengthen the security of your network. Join us in Austin at the fourth-annual Big Communications Event. BCE brings you face-to-face with hundreds of speakers and thousands of industry thought leaders. There's still time to register and communications service providers get in free.


Bill Curtis is someone who seems well suited to answer questions involving command and software quality -- especially software quality. A founder of the Consortium for IT Software Quality (CISQ), Curtis was the leader of the project that created the Capability Maturity Model (CMM) for both software and people. A long-time university professor, Curtis is now senior vice president and chief scientist at CAST and remains a member of the CISQ board of directors.

In a telephone interview with Light Reading, Curtis was reluctant to criticize the software developers at Cloudflare for the incident that became known as Cloudbleed. "There are things that are humanly possible in terms of testing and detection, and then there are things that are just so far out there, they can happen and it's a tragedy when they do, but it's hard to say that they were negligent in their work because it really would have taken some bizarre thinking into the conditions that could occur," he said.

Curtis said that part of the problem of finding the vulnerability is that it did not, in all likelihood, involve a programming mistake. Instead, it was the result of using a parser built on Ragel (not developed in-house by CloudFlare Inc. ) in a very particular, very specific set of circumstances. Within those circumstances, a buffer overflow could occur, and personal information could be released.

The buffer overflow was, according to Curtis, part of what made early detection of a problem so difficult. "Here's the thing about buffer overflows; we don't really do a lot of analysis on buffer overflows and the reason is that there's a zillion false positives -- it just creates havoc," Curtis said. "Some of our competitors go after buffer overflows and they get flooded with false positives."

"For most of these buffer overflows it's really the context that makes that code cause an overflow. And you've got to understand the context, which is not easy. That's a whole 'nother level of analysis and if you read the piece that Cloudflare wrote they listed all the conditions that had to occur," Curtis explained.

"That's a nightmare to go find through static analysis, or even if you're a smart guy," Curtis said, pointing out that there is no reasonable testing regimen that can be expected to find all the issues in complex, modern software systems. "That's the problem we have in software; the incredible complexity that we've gotten into now and the difficulty of detecting these [issues]," Curtis said.

He pointed to a software quality regimen that found an extraordinary number of issues, but went beyond the effort that most organizations can afford -- the detection and testing regiment for the avionics systems on the Space Shuttle. "These guys were at a point where the defects they were detecting were all over ten years old in the code. They weren't generating new defects," Curtis said. "And their analysis, detection, and testing were so thorough, in fact two-thirds of all their effort was in testing."

The professionals in the software development group on Space Shuttle avionics spent much of their time coming up with bizarre scenarios involving anomalies that no one had ever seen, but that were not impossible according to the laws of physics. Commercial developers would have to go into the same sort of imagination exercise to find interactions like the one that led to Cloudbleed. "You'd really have to be thinking, 'What really isn't probably going to happen but possibly could?' If all these different conditions occurred, you'd say that there were all these bizarre little things that had to happen in order for buffer overflow to occur," Curtis explained.

Curtis thinks that the best prospect for avoiding Cloudbleed-like future problems may lie with the computers themselves. "For these things that are context dependent and very tricky, I'm hoping that we can apply machine learning techniques, that maybe the machine learning can go out and begin to understand some of these bizarre contexts and find some of the things that might have been innocent but go on to create some serious problems," he said.

Until machine learning becomes the norm, Curtis believes that rapid response to revealed issues is the practical model for the future, especially since there's no blame to be placed on the development program at Cloudflare. "If this could have occurred frequently then, yeah, they screwed up. But if they couldn't have anticipated the complex set of circumstances required for it to occur then they weren't negligent," he said. "You know, we get a lot of this in complex systems, where people just couldn't have imagined all the interactions that led to the problem. And that's something we're going to live with more and more as these systems get more complex and we have different pieces coming from different vendors."

— Curtis Franklin, Security Editor, Light Reading

(1)  | 
Comment  | 
Print  | 
Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
danielcawrey
50%
50%
danielcawrey,
User Rank: Light Sabre
3/20/2017 | 2:12:51 PM
Too much
It would seem that maybe there shouldn't be an overreliance on Cloudfare. The internet needs to be distributed. I don't think there's anything wrong with the company itself, it just seems like it was more of a central point than many of us realized. 
Educational Resources
sponsor supplied content
Educational Resources Archive
From The Founder
Cisco's Conrad Clemson, recently promoted to head up the company's Service Provider Apps & Platforms developments, talks to Light Reading's Founder and CEO Steve Saunders about how he's bringing cloud video, mobile and virtualization together to empower network operators.
Flash Poll
Live Streaming Video
Charting the CSP's Future
Six different communications service providers join to debate their visions of the future CSP, following a landmark presentation from AT&T on its massive virtualization efforts and a look back on where the telecom industry has been and where it's going from two industry veterans.
LRTV Custom TV
CommScope – Meeting the Demands of Tomorrow's Networks

3|24|17   |     |   (0) comments


Phil Sorksy, Vice President International at CommScope, discusses addressing the challenges faced by service providers today, and as future trends emerge.
LRTV Huawei Video Resource Center
AMS-IX & Huawei's OSN 902

3|24|17   |     |   (0) comments


Huawei shows how its OSN 902 platform helps the Amsterdam Internet exchange to connect the world using multiplexing.
LRTV Huawei Video Resource Center
Huawei's Smart Energy Innovation Center

3|24|17   |     |   (0) comments


In Nuremberg, Huawei showcases its latest capabilities in the digitalization of Internet resources, network infrastructure and intelligence at its Smart Energy Innovation Center.
Valley Wonk
OFC & Hyperscale: A Good Mix?

3|24|17   |   01:50   |   (0) comments


Cloud and telecom players want different types of equipment for their networks, as the chatter at OFC reveals.
LRTV Custom TV
Etisalat on NFV Journey

3|24|17   |   10:37   |   (0) comments


Etisalat is a service provider that prides itself on bringing innovative technologies to the markets it serves. It was one of the first operators to implement 3G and leads the pack in fiber penetration. Now, Esmaeel Al Hammadi, Etisalat's SVP of Network Development, explains the operator's journey to virtualization, beginning with the network core, as well as the ...
LRTV Huawei Video Resource Center
Huawei at CeBIT 2017: Day 3

3|22|17   |     |   (0) comments


Light Reading reports from CeBIT 2017 in Germany, where Huawei is exhibiting on the application of technologies and key business verticals such as transportation, smart city, manufacturing, media and finance.
LRTV Documentaries
No Regrets: Cox's Finkelstein on Fiber & More

3|22|17   |     |   (0) comments


At the Cable Next-Gen Technologies & Strategies event in Denver, Cox's Jeff Finkelstein examines the cable capex conundrum.
LRTV Documentaries
Cable Next-Gen: The 'Mile High' View From Denver

3|22|17   |   11:56   |   (0) comments


Alan Breznick kicks off the Cable Next-Gen Technologies & Strategies event in Denver, casting his thousand-yard stare over cable's current competitive landscape.
LRTV Huawei Video Resource Center
Huawei at CeBIT 2017: Day 2

3|21|17   |   2:27   |   (0) comments


Light Reading reports from CeBIT 2017 in Germany, where Huawei is exhibiting digital transformation solutions around IoT, smart data centers, OpenCloud ecosystem and its newly announced storage-as-a-service solution.
LRTV Custom TV
Driving Better Mobile Customer Experience While Transforming the Mobile Network

3|21|17   |   7:47   |   (0) comments


The Citrix NetScaler mobile gateway is an intelligent traffic management solution which can markedly improve the customer experience provided by mobile operators, even when traffic is encrypted. Critical network services can be consolidated and virtualized using NetScaler. Because of the unique architecture, NetScaler can be deployed on any hypervisor, on a ...
LRTV Custom TV
Mastercard: What's Next for Mobile Payments?

3|21|17   |   7:49   |   (0) comments


2017 marks the fifth consecutive year for Mastercard at Mobile World Congress and it was a great time to reflect on the amazing advances the payments industry has made as well as discuss "What's Next' in the digital commerce future. We spoke to James Anderson, executive vice president of digital payments at MasterCard, about digital wallets to tokenization to ...
LRTV Custom TV
Mastercard: 2 Billion Adults 'Trapped' in Cash Economy

3|21|17   |   03:51   |   (1) comment


Despite advances made in the last several years, two billion adults around the world are trapped in a cash economy and lack what we take for granted -- a safe way to receive, save and use money. Shamina Singh, executive vice president of sustainability and president of the Mastercard Center for Inclusive Growth, chats about how Mastercard is developing new ways to ...
Upcoming Live Events
May 15-17, 2017, Austin Convention Center, Austin, TX
May 15, 2017, Austin Convention Center - Austin, TX
June 6, 2017, The Joule Hotel, Dallas, TX
All Upcoming Live Events
Infographics
With the mobile ecosystem becoming increasingly vulnerable to security threats, AdaptiveMobile has laid out some of the key considerations for the wireless community.
Hot Topics
High-Band 5G: Let's Address the Range Question, Shall We?
Dan Jones, Mobile Editor, 3/21/2017
Eurobites: A1, Nokia Turn It Up to 11
Paul Rainford, Assistant Editor, Europe, 3/22/2017
FTTH No Slam Dunk for Cable
Carol Wilson, Editor-at-large, 3/23/2017
Welcome to the Wild West of Privacy
Carol Wilson, Editor-at-large, 3/24/2017
Like Us on Facebook
Twitter Feed
BETWEEN THE CEOs - Executive Interviews
TEOCO Founder and CEO Atul Jain talks to Light Reading Founder and CEO Steve Saunders about the challenges around cost control and service monetization in the mobile and IoT sectors.
At MWC 2017, Qualcomm's CTO Matt Grob talks to Light Reading's CEO and Founder Steve Saunders about the progress being made in the development of the technologies and standards that will underpin 5G.
Animals with Phones
Neither Do We Click Here
Is that a prerequisite?
Live Digital Audio

Playing it safe can only get you so far. Sometimes the biggest bets have the biggest payouts, and that is true in your career as well. For this radio show, Caroline Chan, general manager of the 5G Infrastructure Division of the Network Platform Group at Intel, will share her own personal story of how she successfully took big bets to build a successful career, as well as offer advice on how you can do the same. We’ll cover everything from how to overcome fear and manage risk, how to be prepared for where technology is going in the future and how to structure your career in a way to ensure you keep progressing. Chan, a seasoned telecom veteran and effective risk taker herself, will also leave plenty of time to answer all your questions live on the air.