Time warp could cause millions of old 'PowerKEY' set-tops to go on the blink
Millions of digital cable set-tops that use the "PowerKEY" conditional access (CA) system originally developed by Scientific-Atlanta decades ago could go on the blink in late 2024 due to an internal computing clock issue that will cause the devices to "time out" and become inoperable.
Boiled down, the internal clock of those boxes – based on a so-called "epoch time" that sets the zero point/starting time in certain computing systems – will effectively roll over in November 2024 because of memory limitations in the secure microprocessor that keeps time. That time rollover won't prevent the underlying PowerKEY security system from operating as designed, but the clock on the box's secure micro will no longer match up properly and will therefore prevent the set-top from decrypting video.
Notably, PowerKEY isn't alone with respect to this kind of computing clock rollover issue. As a prime example, the time-keeping mechanism in certain Unix-based systems is poised for a rollover on January 19, 2038.
The good news is that there's still plenty of time for cable operators to remedy the situation using technical workarounds or swapping out those old, QAM-based boxes.
The not-so-good news? It's not a small fix. Industry insiders familiar with the situation estimate there could be about 20 million PowerKEY devices still operating in the US and Canada that are subject to the time-out issue.
M&A has caused the PowerKEY ecosystem to become fragmented
And getting those fixes developed is also a tad complicated as M&A activity over the years has caused elements of the PowerKEY conditional access system to change hands. Today, certain elements of PowerKEY reside in multiple hands.
Scientific-Atlanta, a video and network tech company that once was a prime supplier to the cable industry, originally developed PowerKey about 30 years ago. That technology changed ownership when Cisco acquired S-A in 2006. Cisco later shed the S-A assets – Cisco sold its set-top business (which included the bulk of PowerKEY) to Technicolor in 2015, and sold its video software unit, which included some control plane elements used to operate PowerKEY, in a 2018 transaction with what would eventually become Synamedia.
Meanwhile, another company, Canada-based Adara Technologies, relies heavily on PowerKEY set-tops to deliver and support video services for dozens of cable operator partners.
Cooking up some remedies
Those suppliers say they're on the case. Vantiva, a company recently spun-out of Technicolor that makes set-tops and possesses most of the PowerKEY assets, alerted operators to the issue on June 1, 2022.
PowerKEY-enabled products, including both video set-tops with embedded security and those with CableCARD security modules, "will cease to perform decryption of video in the August 2024 timeframe," Vantiva explained in the letter, which was obtained by Light Reading. "This will result in no video being displayed."
At the time, Vantiva recommended that operators remove those devices from their networks by August 2024, but noted that some of the more recent models might be saved "via an update to the client conditional access software image." Since those products are now out of warranty and reached end-of-support several years ago, Vantiva also indicated that such fixes won't be free. Vantiva also suggested that operators consider new IP-based set-top technology as a potential replacement.
Jim Potts, SVP of sales for North America at Vantiva, said the company has pulled in some people with knowledge of PowerKEY off other projects to help resolve the issue – at least for devices that use CableCARD security modules. Those devices could include operator-supplied set-tops originally made by the likes of S-A, Cisco and Technicolor, certain TiVo DVRs and the few HD televisions equipped with CableCARD slots that remain in existence.
Vantiva is "very far along with having a solution" for PowerKEY devices that use CableCARDs, Potts said. He stressed that operators with PowerKEY boxes in the field that haven't been alerted to the issue should contact Vantiva as soon as possible.
"The beauty about a CableCARD is that it's disconnected from the host [set-top], so all you have to do is get a message to the CableCARD. There's a way to basically reengineer how some of the things work within PowerKEY so that it can handle this rollover date."
Potts said work is underway in a few customer labs and that Vantiva expects to have a solution available in early 2023. Vantiva, he noted, is working on that solution with Synamedia, since the PowerKEY control plane will also need to account for the rollover fix. Synamedia has been asked to describe its role in developing this solution, but, as of this writing, has failed to offer any detail.
Update: Synamedia confirmed that it's lending a hand. "At Synamedia, we are committed to supporting PowerKEY users. We are already working with customers to upgrade the control plane to ensure that it will continue to work with PowerKEY devices," Cyrille Berson, market segment director of edge delivery at Synamedia, said in a statement.
PowerKEY set-tops that use embedded security (i.e. no CableCARD) are "much more of a challenge," Potts acknowledged. In those cases, a fix would involve the entire host device, and some older models might not even possess the original code.
The good news is that those boxes represent a small minority of PowerKEY devices still in the field, Potts said. Many of those embedded security boxes were likely made and deployed prior to the FCC's old separable security rules that originally took effect in July 2007 and have since been abolished. In those instances, Vantiva is suggesting that operators swap them out for a new device.
Adara pitches an alternative approach
Adara, which operates PowerKEY boxes and runs its own video control plane platform for dozens of North American cable operators, is also developing its own clock rollover fixes.
In addition to talking to customers that use Adara's managed platform about those options, Adara is also having similar discussions with other cable operators that are grappling with the clock rollover issue, according to company CEO and co-founder Joseph Nucara.
Adara has some skin in the game – the company has "millions" of boxes in operation that are subject to the 2024 PowerKEY time warp, according to Nucara.
Once Adara was alerted to the issue, he said the company's engineers replicated the scenario in its own labs and figured that the exact time-out date will occur on November 24, 2024, based on a PowerKEY epoch/start time of January 1, 1993.
"It's almost like the Y2K situation, except this one's real," Nucara said, referring to the so-called "Y2K bug" that was cause for concern for older computers that did not use four-digit years in their internal calendars as the year 2000 approached.
Nucara believes Adara has a better proposal that would cover all generations of PowerKEY, including devices that use embedded security, CableCARD modules or a downloadable form of PowerKEY. That approach, he said, would prevent operators from being saddled with costs and potential customer disruption associated with device swap-outs.
Adara, he said, has developed some "procedures and scripts in the background, that on an operator-by-operator basis, can basically spoof that time clock in the system." By resynchronizing the entire ecosystem, Nucara explained, PowerKEY boxes can continue to decrypt video and ensure that various set-top box applications, such as guides and pay-per-view and video-on-demand services, continue working.
"The time-warp issue does not happen at all. That becomes completely transparent," he said.
Nucara said Adara has developed the required technology and has it working in the lab. "We are delivering hard, firm proposals to operators," he added.
As for operators that already use Adara's managed platform, the PowerKEY fix will be implemented automatically with the general support that the company is already providing to those customers, he said.
Nucara acknowledges that operators can also sidestep the issue by migrating to IP-based video devices and platforms. But he warns that such a move is not risk-free either, as a forced migration could result in some video subscriber losses (Nucara believes that to be in the neighborhood of 5%), and possibly cause some customers to get broadband services from a competing service provider.
How Comcast and Charter are responding
Examples of North American cable operators known to have PowerKEY boxes still in the field include Altice USA, Astound Broadband, Breezeline, Comcast, Cox Communications, Charter Communications, Rogers Communications and Videotron.
It's not yet clear what all of them intend to do in order to iron out the issue over the next couple of years. However, the two largest US cable operators – Comcast and Charter Communications – say they are well aware of the problem and already have remedies in the works.
Comcast, an official says, is working on a plan to swap out impacted boxes in 2024, but didn't say how many set-tops are presently subject to a PowerKEY-related change-out.
Charter tells Light Reading that more than 99% of the PowerKEY boxes it currently has in the field can be resolved remotely. As for that small number of remaining PowerKEY boxes, Charter plans to communicate with affected customers in 2023 about what, if anything, they'll need to do in order to continue receiving video services.
Legacy video tech starting to fade out
Meanwhile, Charter's time with legacy security technologies such as the CableCARD continues to near the end of its course. The operator reportedly has issued warnings that the security modules won't be compatible with some coming network upgrades. That's likely due to the fact that out-of-band frequencies used by CableCARDs won't be compatible with coming "high-split" upgrades that will vastly expand the amount of upstream spectrum supported by Charter's hybrid fiber/coax (HFC) network. Charter is working out a solution that will allow CableCARDs to function after those upgrades, but in the meantime is encouraging those customers to lease a new set-top box or to use its pay-TV streaming app for devices such as the Apple TV.
A Charter official told TechHive that a "fraction of 1%" of its pay-TV subs currently use CableCARDs. That base likely includes a few customers who use TiVo boxes or maybe some older CableCARD tuning devices from companies such as Ceton and SiliconDust. Charter's reliance on CableCARD devices has also dwindled amid the operator's use of a downloadable security system that works with the operator-supplied, hybrid QAM/IP "Worldbox."
Breaking free of legacy set-top security also ties into the newly branded "Xumo" national streaming joint venture forged earlier this year by Charter and Comcast.
- Vantiva CEO on life after the Technicolor separation and spin-out
- House Votes to Kill CableCARD Mandate
- Ceton Pitches Cable Set-Top Alternative
- The CableCARD's 'Greatest Hits'
- Comcast, Charter brand national streaming joint venture as 'Xumo'
— Jeff Baumgartner, Senior Editor, Light Reading
Like what we have to say? Click here to sign up to our daily newsletter
Cable Next-Gen Technologies & Strategies – Making cable faster, broader, deeper, better
If you want to know where the cable tech space is heading, we've got you covered. Join the biggest names in the industry in Denver, CO on March 14 & 15 for the latest edition of Light Reading's Cable Next-Gen Technologies & Strategies conference.
Back for a record 16th consecutive year, Cable Next-Gen is the premier independent conference covering the broadband technology market. This year's edition will tackle all the top tech topics, including 10G, DOCSIS 4.0, Distributed Access Architecture, next-gen PON, fixed wireless access, network virtualization, the Digital Divide and more
Get your free operator pass here.