Light Reading
Are communications service providers boiling in an ocean of data?

The 'Big Data' Challenge

Column
Column
Column
12/5/2011
50%
50%

"Big Data" was a much-discussed concept in IT management circles last year. But the effect of Big Data -- the non-stop massive growth of information -- has been a challenge for service providers' data infrastructure and technology teams for quite some time.

Tier 1 providers have experienced billions of call-detail records (CDRs) every day for years. In response, custom database technologies are now being developed and refined. Petabyte-scale database systems holding years of call-detail records and Internet protocol detail records (IPDRs) are now considered the norm at many Tier 1 providers.

The Big Data effect has proven even more complex and challenging with the advent of more stringent compliance regulations. A single global provider must proactively configure, manage and retain specific subscriber records for a designated time period. Additionally, the regulatory regimes of different nations impose different timeframes, requiring different data types in different formats. Specific response times to access that data vary across borders as well. In the past year alone, Japan and India have imposed new regulatory initiatives upon providers, regardless of size and subscriber level.

Fulfilling lawful intercept requirements demands substantial investments by providers in storage, database and search technologies -- and, of course, trained personnel with the skills necessary to deploy these solutions. Lawful intercept involves both real-time monitoring and access to data records. For example, law enforcement may obtain judicial warrants to tap a suspect's landlines, cellphones and email accounts, and procure copies of their call, SMS/MMS and other communication records. Having this data readily available in the correct format and delivered in a timely manner creates ongoing pressure to make the data center run efficiently and cost effectively. The possibility of legal fines only compounds the pressure.

Today communications service providers (CSPs) must innovate to achieve scale and better IT economics. Interoperability with other global partners, providers and suppliers is important but not enough. The CSP must be prepared to use technology, network architecture and standardization to deal with these pressures.

A recent Gartner industry report, Predicts 2011: CSPs Must Rethink Business Paradigms to Meet Market Challenges, cited falling margins of CSPs in mobile and fixed combined services sectors in development markets. There are very few exceptions to this trend. The causes of the drop include Internet Protocol (IP) substitution, lower average revenue per unit (ARPU) from new customers, increasing competition, regulators getting a better view of non-competitive price elements and the need to invest in expensive broadband networks with long payback cycles.

Improving the customer experience and service level is absolutely a necessary part of staying ahead. New partnerships are formed daily to provide new applications and mobile capabilities to drive new channels of revenue. However, all of the front-end customer-facing products and services that are rolled out will only be sustainable if back-office and data infrastructure are capable of robust management and increasing scale in the coming years.

A service provider IT issue
Let's consider the data problem from the perspective of today's IT team.

Within the life-cycle of a single customer's communication "transactions" in a given billing cycle, core data (essentially call data detail records and Internet Protocol detail records) that are captured, collated and stored are immediately historical in nature -- once the "transaction" takes place, the data does not undergo any further change. This is quite different from traditional business transactions that are captured in an online transactional database system, such as a retail purchase. This customer communication data is then processed by a series of applications that allow various IT and business functions, including network capacity planning, traffic analysis, user trending, revenue assurance, billing, digital advertising and more.

Some of these OSS and BSS applications leverage the same database system; in other cases, the data is moved around with varying degrees of latency. Data sub-sets are formed depending on application function and requirements, and data warehouses are fed with customer activity in order to perform trending analysis and determine the overall value of a customer at any point in time. Currently, many CSPs retain much of this data in a traditional database such as Oracle. For many of these functions, the data does not need to change; therefore it can be stored in a database whose primary purpose is long-term retention.

Given the growth in data volume and complexity across diverse systems and applications, organizations are forced to retain the data for as little time as possible, largely because systems have grown bloated -- to the point where the sheer size of the data harms performance. For a large provider, even the critical function of replicating data from a complex billing application in order to provide continuous uptime can cause significant technology challenges due to the volume of the data sets.

Specialized databases
However, new database technologies are emerging that promise to solve this problem. Specialized databases with the ability to significantly de-duplicate and reduce this data to a much smaller footprint are being deployed by providers. When you consider a 100-terabyte system reduced to a storage footprint of 5 terabytes, you can begin to imagine the cost savings not only in storage but also hardware capacity. Imagine a petabyte environment reduced to 100 terabytes. These specialized data-retention databases -- built for the purpose of storing data online for many years -- can also perform fast queries using standard SQL or various business intelligence and analytics solutions. With a database solution focused on Big Data retention that enables significant cost reduction, CSPs can store critical customer activity data for much longer periods -- even beyond what is required by regulators. By having access to larger historical data sets, CSPs can gain greater insights into customer behavior and even discover new opportunities to further segment the market and create new services with a wider range of price points.

Today's R&D teams now employ data scientists who dissect and mine usage data to detect behavior patterns in order to predict key market drivers such as what causes churn. Being able to predict customer behavior is much more powerful to the business than analyzing what happened and why it happened after the fact! Once churned, it costs the business as much as six times more to win that customer loyalty back. Those data scientists don't need SQL statements or business intelligence tools -- they must have continuous access to the raw data, the detailed historical xDR data, in order to determine market and behavior patterns. Giving them a dedicated Big Data retention database is the best way to achieve that. By looking inward and discovering new technologies and new approaches to managing core customer and transaction data, CSPs can discover new service offerings -- even new markets.

The ocean of Big Data faced by today's CSP does not have to reach a boiling point. Applying innovative data-retention database capabilities across the infrastructure will significantly reduce cost, and help manage scale and complexity.

— Dr. Hossein Eslambolchi is the former CTO of AT&T and chairman and CEO of 2020 Venture Partners

(0)  | 
Comment  | 
Print  | 
Newest First  |  Oldest First  |  Threaded View
More Blogs from Column
Norway's Lyse Smart is showing how broadband-enabled applications can improve the lives of elderly people and provide a cost-effective alternative to traditional care home services.
If we can build complex systems from simple components with precise functionality, we can more easily change those systems as new technologies arise.
A list of 10 considerations for municipalities pondering building their own broadband networks.
Communications service providers need to become digital service providers, but what exactly does that entail?
Here are some ideas for how cable operators can help low-income households connect to pay-TV and broadband services.
Flash Poll
From The Founder
It's clear to me that the communications industry is divided into two types of people, and only one is living in the real world.
LRTV Custom TV
Advantech Accelerates 100G Traffic Handling

10|17|14   |   7:56   |   (0) comments


Paul Stevens from Advantech explains why handling 100GbE needs a whole new platform design approach and how Advantech is addressing the needs of equipment providers and carriers to give them the flexibility and performance they will need for SDN and NFV deployment.
LRTV Huawei Video Resource Center
Holland's Imtech Traffic & Infra Discusses Huawei's ICT Solution and Services

10|16|14   |   4:49   |   (0) comments


Dimitry Theebe is from the business unit at Imtech Traffic & Infra which delivers communications solutions for transportations. His partnershp with Huawei began about a years ago. In this video, Theebe speaks more about this partnership and what he hopes to accomplish with Huawei.
LRTV Huawei Video Resource Center
Huawei's Comprehensive Storage Solutions Vital for SVR

10|16|14   |   6:16   |   (0) comments


SVR Information Technology provides cloud services for academic and special sectors. With Huawei's support, SVR and Yildiz Technical University has established Turkey's largest and most advanced High Performance Computing system. CSO Ismail Cem Aslan talks about what he hopes Huawei's OceanStor storage system will bring for him.
LRTV Huawei Video Resource Center
Mexico's Servitron's Impression of Huawei at CCW 2014

10|16|14   |   6:35   |   (0) comments


Servitron is a network operator in Mexico that has been in the trunking industry for the past 20 years. Its COO, Ing. Ragnar Trillo O., explains at Critical Communications World 2014 that his company has been interested in the long-term evolution of LTE technology and its adoption for TETRA.
LRTV Huawei Video Resource Center
Building a Better Dubai

10|16|14   |   2:06   |   (0) comments


Abdulla Ahmed Al Falasi is the director of commercial affairs, a telecommunications coordinator for the government of Dubai. Their areas of service span across multiple industries, including police, safety, shopping malls and more. In this video, Abdulla talks about his department's work with Huawei.
LRTV Huawei Video Resource Center
Huawei Lights Up Malaysia Partner Maju Nusa

10|16|14   |   1:59   |   (0) comments


Malaysia's Maju Nusa is an enterprise partner to Huawei in networking, route switches and telco equipment. At this year's Critical Communications World in Singapore, CTO Pushpender Singh talks about what Huawei's eLTE solutions mean to his company and for Malaysia.
LRTV Custom TV
Evolving From HFC to FTTH Networks

10|15|14   |   2:19   |   (0) comments


Cisco's Todd McCrum delves into the future of cable's HFC plant, examining how DOCSIS 3.1 and advanced video compression will extend its life and how the IP video transition will usher in GPON and EPON over FTTH.
LRTV Custom TV
Exploring the Future of Cable Access

10|15|14   |   6:23   |   (0) comments


Cisco's Brett Wingo looks at where cable access architectures are heading, discussing the impact of DOCSIS 3.1, CCAP, Remote PHY, SDN, virtualization of cable networks and related technologies.
LRTV Custom TV
Optimizing & Monetizing WiFi

10|15|14   |   5:53   |   (0) comments


Cisco's Vince Pandolfi outlines the reasons for cable WiFi's rapid growth, lays out the issues with the technology and explains the new Cisco tools that can help operators monitor and improve their WiFi delivery.
LRTV Huawei Video Resource Center
Emulex & Huawei Collaboration Mutually Beneficial

10|14|14   |   4:17   |   (0) comments


US company Emulex collaborates with Huawei in areas such as blade servers and workload documentation. Mike Heumann of Emulex believes that Huawei has done incredibly well moving from a telecom company into servers and networks, working closely with customers to realize their needs.
LRTV Huawei Video Resource Center
Chile's VZION Looks Forward to Seeing More of Huawei

10|14|14   |   4:43   |   (0) comments


VZION is a systems integrator company in Chile with a focus on virtulization technology. In this video, Cesar Alcacibar talks about the challenges in virtualization and how Huawei helps his company to achieve the best results possible. Alcacibar is expecting more adoption and integration of Huawei technologies in Chile.
LRTV Huawei Video Resource Center
Pro-Datech & Huawei for a High-Tech Singapore

10|14|14   |   2:59   |   (0) comments


Pro-Datech Systems is a specialty IT solution provider based in Singapore. For an added value to its customers, the company uses Huawei's hardware and trusted performance and features for a total solution. It's looking forward to the creation of a lab, to be based in Singapore, for the two companies' coorporation on total storage solution.
Upcoming Live Events
October 29, 2014, New York City
November 6, 2014, Santa Clara
November 11, 2014, Atlanta, GA
December 2, 2014, New York City
December 3, 2014, New York City
December 9-10, 2014, Reykjavik, Iceland
June 9-10, 2015, Chicago, IL
Infographics
WhoIsHostingThis.com presents six of the world's most extreme WiFi hotspots, enabling the most epic selfies you can imagine.
Hot Topics
Forget the Internet, Brace for Skynet
Stephen Saunders, 10/15/2014
HBO Will Go OTT in 2015
Mari Silbey, Independent Technology Editor, 10/15/2014
Google: Carriers & Cloud Providers Need to Cooperate
Mitch Wagner, West Coast Bureau Chief, Light Reading, 10/16/2014
CBS Takes OTT Plunge
Mari Silbey, Independent Technology Editor, 10/16/2014
Could Data Be the New 'Currency'?
Ray Le Maistre, Editor-in-chief, 10/16/2014
Like Us on Facebook
Twitter Feed