the application of data science to those feeds to automate a response to low-level threats and allow concentrated forensic analysis on security incidents that are -- or appear to be -- most threatening.

a means of extrapolating exactly what pre-emptive adjustments are required to the enterprise's security posture in order to strengthen it against newly identified threats.

a means of rapidly importing security change recommendations arising in software into the enterprise's workflows.

The art of bringing a high-value threat intelligence capability to market consists of the application of data science and human intervention to the raw threat feeds. It is this filtering and curation which enables the vast amount of threat data to be ignored or else responded to very quickly.

It is then the same filtering and curation function that allows for the most suspicious data to be extracted from the main body of the threat data. The SecOps team's resources can then be concentrated on applying greater forensic effort around that data subset in an effort to understand the modus operandi of the most threatening adversaries -- and stay ahead of them.

This is a primary area where threat intelligence providers differentiate themselves. Machine-learning algorithms leveraging standard and advanced statistical models -- and customized to cybersecurity goals -- have to be used to automatically process the many billions of security events that threat intelligence providers see.

Big data algorithms are the core engine that drive the critical automation component of threat intelligence. Without this automation, large teams of cybersecurity professionals would have to paw over these vast data sets themselves, dedicating their time to working on security events which don't actually pose a significant threat.

It is these key individuals in the security team that do the most important work in threat intelligence. They do it by leveraging the big data algorithms themselves, combining their outputs with human intelligence gathered on major threat actors, and then layering in their own assumptions. This enables threat intelligence analysts to correlate suspicious events with other sources and spot patterns that the big data engines themselves might not spot.

The marked shortage of cybersecurity professionals relative to growing demand is well known. Last year the CEO of Symantec, Michael Brown, estimated that there will be a global shortfall of these key people amounting to 1.5 million by 2019. Given some of the skillsets required, as well as the highly rewarding nature of the role serving in the front line of cybersecurity, threat intelligence is an area where the competition for talent is at its fiercest.

To be competitive, any threat intelligence provider needs to offer opportunities, challenges and compensation packages that are fit for individuals that comprise some of the cream of top cybersecurity talent. These individuals will always want to be working at the very cutting edge of monitoring, anticipating, foiling and disrupting criminal cyber adversaries -- and they will go wherever those opportunities are to be found.

Organizations that can't offer that kind of stimulating environment lack the basic platform on which to build long-term competitiveness in the threat intelligence space. Those that can are very much better placed to succeed.

This blog is sponsored by AT&T.

— Patrick Donegan, Founder, HardenStance and Contributing Analyst, Heavy Reading