Docker Users: Tales of Triumph & Woe

Craig Matsumoto
4/25/2017

A common theme among enterprises speaking at DockerCon last week was that while Docker containers have proven their usefulness, they're also still a work in progress.

Speaker sessions were slanted toward success stories, of course, but they also provided some lessons based on fresh wounds.

Intuit Inc. (Nasdaq: INTU), for instance, learned that applications have to behave differently in a container environment. The company started using Docker in earnest in mid-2016, starting with one project that had careful boundaries -- no container networking allowed, for instance.

That created other problems. Lacking container networking, each Docker instance had to keep contacting a router, and it turned out Intuit's applications did this way too often. That caused baffling connection resets.

"We had to go to work on our applications," Lahpoor said. "The applications had to get used to more volatile environments."

Intuit also set up containers to automatically update their operating systems -- which, it turned out, had the capacity to make DNS go haywire inside a container. That created a "zombie container" that wouldn't respond to any requests.

Intuit's case has an interesting twist: tax season. Internal applications start drawing lots of traffic in November and December, and it sounds like that's when a lot of the company's problems came to light. Christmastime was particularly unhappy, Lahpoor said.

"If you have a project that's already behind schedule, that is not a good candidate to Dockerize," he said. "This is only four years old. You can't expect it to run like VMware."

Preachin' Docker
Jim Ford of ADP shows that big, old enterprises really can learn to love containers.
Jim Ford of ADP shows that big, old enterprises really can learn to love containers.

For Visa, which put its first Docker applications into production in late 2016 after about a year of development, the issue was managing the lifecycle of containers.

The company's core systems have run with zero downtime for two decades, Chief Systems Architect Sasi Kannappan said, so high availability was a major concern. That meant constantly pinging containers to check status, and Visa had to write some of its own code to handle that. Docker 1.12, in 2016, helped by including a health-check function for containers.

Given all these troubles, why do any of this?

Oddly, the biggest champion for Docker was one of the oldest and most staid enterprises to present: Automatic Data Processing Inc. (ADP), which offers software for payroll and other human resources functions. ADP spoke at last year's DockerCon, and Jim Ford, chief architect, came back this year for an update.

Most enterprises talk about "developer productivity" as a reason to us containers. ADP was able to confirm that, noting that developers have more power to see what's happening with their code.

"It helped us get developers into more of what I would call a flow state," he said. "They kind of stay in that coding mindset while they work through [a problem] and get that immediate feedback."

Containers also make it faster to move applications into production from test. "We were tired of this complex deployment cycle that would take the better part of a Sunday and roll into Monday," said Shawn Bower, cloud architect at Corning University.

Security is an interesting side effect of containers. They're small and meant to be ephemeral, so they don't linger as tempting targets.

"Get to where your stuff is short-lived, because then the bad actor needs to get in and re-compromise you," Ford advised the audience. "If you're churning images every month, you've reduced the compromise window to 30 days."

At the same time, Ford has a security concern: Are the images available on Docker Hub secure? Docker certifies the containers on the hub, which anybody can download and use, but that's not enough assurance for an $11 billion-a-year company like ADP. "I'm not risking $11 billion on Docker telling me it's safe," he said.

— Craig Matsumoto, Editor-in-Chief, Light Reading

(8)  | 
Comment  | 
Print  | 
Newest First  |  Oldest First  |  Threaded View        ADD A COMMENT
kq4ym
kq4ym
5/5/2017 | 12:20:08 PM
Re: Security by obscurity...
Interesting how the use will be postive for some security issues in using containters in that "They're small and meant to be ephemeral, so they don't linger as tempting targets. That alone might be worth it's weight in security gold.
Joe Stanganelli
Joe Stanganelli
4/29/2017 | 7:02:50 PM
Re: Security by obscurity...
@mhh: It's all about the complementary technologies that go with containers, too -- especially microservices and other virtualization techs.
mhhfive
mhhfive
4/25/2017 | 6:18:06 PM
Re: Security by obscurity...
> "Docker containers have proven their usefulness, they're also still a work in progress."

Clearly, containers are useful for certain things, but they're not general to every problem.. yet? Right now, there are too many custom uses, but I assume that will change as more and more developers try it out and start to aggregate solutions in a repository.... 
danielcawrey
danielcawrey
4/25/2017 | 3:59:08 PM
Re: Security by obscurity...
Testing environments can become very complicated very quickly. This is why I've always thought container technology is useful. Everyone is going to be using this stuff. It just takes a long time. Good get for ADP to be involved, that's for sure. 
mhhfive
mhhfive
4/25/2017 | 2:42:37 PM
Re: Security by obscurity...
I think we're in agreement that there are no silver bullets for security. There may be some temporary hideouts from malicious attackers, but if the targets are worthy, it's only a matter of time before creative bad guys figure a way in. Security by obscurity is one of the nicest temporary hideouts, though.
JohnMason
JohnMason
4/25/2017 | 2:39:24 PM
Not a fatal flaw
Sounds like traditional transitional turbulence, but not a fatal flaw.
Joe Stanganelli
Joe Stanganelli
4/25/2017 | 1:49:10 PM
Re: Security by obscurity...
At the same time, containers have their own security flaws.  If the shared kernel becomes compromised, so too do every single one of those containers.  Moreover, a breach of a single container could bleed over to compromises of additional containers.

Unikernels can resolve these security concerns in terms of both actual security as well as scalability -- although they have their own practical issues themselves.
mhhfive
mhhfive
4/25/2017 | 11:27:52 AM
Security by obscurity...
> "Security is an interesting side effect of containers. They're small and meant to be ephemeral, so they don't linger as tempting targets."

That's an interesting promotion of security by obscurity. It sorta works, but if it really worked that well... why not constantly rotate containers to keep bad guys at bay?  :P
Featured Video
Upcoming Live Events
October 22, 2019, Los Angeles, CA
November 5, 2019, London, England
November 7, 2019, London, UK
November 14, 2019, Maritim Hotel, Berlin
December 3-5, 2019, Vienna, Austria
December 3, 2019, New York, New York
March 16-18, 2020, Embassy Suites, Denver, Colorado
May 18-20, 2020, Irving Convention Center, Dallas, TX
All Upcoming Live Events