Hitachi Consulting is touting a big data analysis product that will enable network operators to do something with the oceans of data their networks generate: proactively improve their quality of service (QoS).
Hitachi Consulting says its Live Insight can take input from anything on the network that generates data -- from network gear to probes and monitors, and evaluate that data in real time. Furthermore, it can do so with extraordinary specificity.
The tool will not only tell you that you dropped a packet, said Hitachi Consulting vice president Kerry Sims, it'll tell you which packet was dropped.
Thus far, network operators have been immune to the appeal of big data. By the time big data tools can sift through bogglingly vast amounts of network data to identify a problem, it's either too late or it wasn't that big of a deal anyway.
But that's because most big data systems don't work in real time. Hitachi warrants that its Live Insight operates in real real time (as opposed to near real time, or close enough to real time), enabling network operators to identify and solve incipient problems before they manifest themselves.
Any problems that network operators might experience demonstrably never justified the investment in developing real-time data analysis tools, but Hitachi's Live Insight was developed for another reason entirely: counteracting the distortions in global markets caused by high-frequency trading (HFT).
Several years ago, the Tokyo Stock exchange asked Hitachi to develop such capabilities to help figure out some anomalous behavior in global financial markets associated with HFT. Michael Lewis explained the situation in his book Flash Boys , in which he asserted that some traders were deliberately manipulating the market. Others refute that, but there's general agreement that there were serious distortions in the market, at least partly attributable to the inability of communications networks to keep up with HFT.
The only way to keep up with what was happening was with real-time big data analysis. At the Tokyo Exchange's behest, Hitachi built the tools (neither the Tokyo Exchange nor Hitachi are mentioned in Flash Boys). Those tools are the basis for Live Insight.
So the development work is done; now it's time to find new applications. Hitachi was at the recent Cable-Tec Expo conference to tout Live Insight. Sims said the tools can be used to keep on top of network performance parameters, such as latency and jitter, and as the cable industry moves forward with its Energy 2020 project, big data analytics could play a role in network management.
— Brian Santo, Senior Editor, Components, T&M, Light Reading