AT&T this week unveiled a new powerful security platform, using big data analysis based on a Hadoop architecture which allows the company to ingest and analyze 5 billion security events in less than ten minutes.
Developed over the last three-and-a-half years as part of the Domain 2.0 push to SDN, the new threat intelligence platform collects data from every router, switch and server in AT&T's network and uses machine learning to constantly analyze and correlate data to detect threats faster. (See AT&T Intros New Security Service Suite.)
All of this is underlying the AT&T Threat Intellect, the name the company has given the people, processes, products and tools that are part of its security operation. And while Jason Porter, AT&T Inc. (NYSE: T)'s VP of security, won't say that this new security engine helps the company stay ahead of the bad guys, he is confident the company can detect threats much earlier and limit their impact on customers.
"This is reducing the time it takes us to deploy security capabilities by greater than 95%," Porter tells Light Reading in an interview. "It allows us to be much more accurate, because we can correlate many different signatures. And more than it ever has before, the machine learning can identify that there is an abnormal traffic pattern and, with high confidence, detect a correlated threat event."
That includes events within AT&T's network but also within its customers' data centers and on their devices, including mobile devices, where AT&T is acting as security agent. "We protect their data at rest and in motion," he says. "And we are identifying malware on their devices -- all of that feeds into the threat platform."
AT&T committed to building this new security platform as part of its shift to a software-defined network, and the Hadoop architecture was built from the ground up to be a security system, Porter says. It can handle the 117 petabytes of traffic that crosses AT&T's network by analyzing the large chunks referenced above much faster than human security scientists could possibly do, freeing up their time for other work, such as improving how the machine learning is informed, he notes. They can zero in more quickly on new behavior that the analysis identifies and determine what is a new threat, feeding that intelligence back into the platform.
"Our PhDs and data scientists can focus on new things -- they are always going to be reshaping and changing their attack rules, what this allows us to do is identify those more rapidly," Porter says. "We made the commitment to make security foundational, at the time we did Domain 2.0," he comments. "So simultaneous to doing software-defined networking, we agreed that everything had to feed [data] into this platform."
One advantage to using constant analysis of big data and the machine learning is that threats can be detected when the volume of traffic is very low -- much too low to trigger the kind of threshold alarms commonly used in the past, Porter notes. An anomaly that impacts less than 1% of traffic can still be noted and examined for its threat level in the specific context of the affected traffic and its past behavior. Formerly, that kind of event might not be noticed until it was impacting much more traffic and potentially impacting customers.
AT&T has been introducing the new capability in stages, beginning about a year ago when it was used to start protecting the AT&T network end points and its internal enterprise network. "Since then we have been curating the data, making sure the machine learning was accurate and doing what it needed to do," he notes. Part of that process was connecting to all the firewalls within the network and building up an applications programming interface platform that ingests all the data.
That layer of APIs surrounding the security platform also lets AT&T connect to and share information with its customers, and through them share industry-specific knowledge gained with key customer segments, such as banking, healthcare and government. Bi-directional sharing of data with customers who also invested in big data and analytics for their security operations lets the systems get smarter faster.
"We become two sets of eyes examining that data," Porter says.
AT&T is not directly sharing its information with other ISPs, but it is sharing with the broader security community, including hardware and software vendors that are part of its cohort.
There is a competitive differentiator for the company here with its enterprise customers, as it bulks up the quality of the security services it delivers from the platform in an increasingly competitive market for managed security services. The new platform helps fuel an approach to security that moved away from a perimeter approach to wrapping applications or workloads in their own security.
— Carol Wilson, Editor-at-Large, Light Reading