So now we're down to brass tacks: The comparison of what the Broadband Forum did to create TR-69 and what OPNFV is doing with NFV infrastructure.
For a telecom standard, TR-69 actually came together fairly quickly, most likely because it addressed a specific and universal pain point for broadband ISPs -- it's still widely used today, by the way. The group that assembled within the forum to address the issue -- which started out as the auto-config working group, says Kirksey -- spent 18 months arguing over how to accomplish what everyone agreed needed to be done. But after lengthy discussions, including some meetings that ended in the bar, where beer helped fuel the collaborative spirit, the group had -- on paper -- the specifications for how TR-69 could work to enable remote diagnostics and management of CPE.
But that's when things got interesting, Kirksey notes -- and by interesting, she means a bit contentious. The paper spec was given to the engineers at each of the companies involved in the spec at the time, and there were many. Two things happened after that: Either the engineers found the specs incomplete or different people at different companies interpreted them differently and built differing solutions on the same specs.
So the first 18 months of hard work was followed by another 18 months of painful work because implementations of the spec had started but they weren't interoperable, which was a baseline requirement of the broadband operators.
"We had to do a lot in the weeds, a deep dive, painful technical work to get to that next appropriate level," Kirksey says. "People who had invested engineering resources in doing things one way had to go home and tell their engineers 'You have to redo it.' "
Plugfests to the rescue
What ultimately produced the spec that works well today was a series of plugfests at the University of New Hampshire InterOperability Laboratory (IOL) which produced results -- kept confidential at the time so vendors would continue to share information -- that were fed into the Broadband Forum working group, until "every single point of dissent or lack of interoperability" was addressed, says Kirksey, who was co-chair of the group at the time.
She knows this because, along with 2Wire Cofounder Jeff Bernstein, Kirksey created a spreadsheet that tracked every bug, every interop-related issue and every plugfest result. "We made sure each one was addressed and each bug and interop-related issue was fixed," she recalls.
Relationships became a key part of the process -- figuring out who was intent on getting work done and who was a political grandstander, and understanding when to defer to a participant because of their specific expertise, were both essential factors, she says. Kirksey credits TR-69's staying power to the group's determination to work through even the "big ticket items" to deliver true interoperability.
Next page: How it works in open source