This content is part of the Essential Guide: The complete rundown on Docker data storage and containers

Docker containers expected to emerge gradually in primary storage

Storage admins may find the container concept familiar, even if the terminology seems new; in some ways it's similar to using VMs.

Since their inception two years ago, Docker open source containers have been used mostly in application development...

and testing environments. How long before Docker containers are routinely deployed in primary storage is tough to forecast, but experts see a parallel to VMware, which gradually added features for creating, provisioning and managing storage to its server virtualization software.

"The key to Docker from a storage perspective is you really need a storage system that provides a lot of capabilities," said George Crump, president of IT analyst firm Storage Switzerland. "Like VMware in the early days, the storage aspects of Docker are pretty rudimentary. There's a dash-V parameter that allows you to mount images from a storage volume, but that's about all it does."

Crump said VMware has "already ploughed the field" for Docker to make its way into primary storage systems, although it may be a slow process.

"My expectation is that Docker initially will be used in storage environments where you have a lot of the same instances. Say you're rolling out 80 Oracle or SQL Server apps. The efficiency and capacity should be high because you're not recreating all the operating systems within those apps," Crump said.

Docker storage similar, yet different, from VM storage

Docker is open source software for packaging a Linux application and its dependencies with a shared operating system. The buzz surrounding Docker storage containers thus far has exceeded widespread enterprise adoption of the technology.

But Crump and other industry observers say it's more a matter of when, not if, containers become a complement to virtual machine storage.

Docker Inc. commercialized the open source Docker container, but competitors are emerging, including Docker partner turned rival CoreOS with its Rocket container runtime engine. Google also last year unveiled Google Container Engine based on its Kubernetes open source container cluster management framework.

EMC, Asigra, Catalogic and Zadara are among storage vendors to roll out product support for Docker containers. Microsoft said it plans to support Docker beginning with the rollout of Windows Server 2016.

Rob "Bubba" Hines, vice president of IT at Asheville, NC-based Signature Tech Studio, said his company runs thousands of jobs a day in Docker containers on Amazon Web Services. He said he is looking forward to taking advantage of Zadara's Docker support inside its Virtual Private Storage Array (VPSA), which Signature Tech uses as its main storage platform. But that will be a gradual process.

"Dockers have impacted the way we think about our architecture and infrastructure, but as far as our storage goes, a Docker instance is like any kind of virtual machine," Hines said. "It can access Zadara. Initially we'll run a handful of Docker containers inside the VPSA. I don't know if I'm ready to move hundreds or thousands into the VPSA, but maybe that's where we'll end up."

Docker containers bear similarity to virtual machines (VMs), but are also distinctly different.

"The biggest difference between VMs and containers is granularity," said Greg Schulz, founder of the StorageIO Group. "I could run five or 10 different VMs on a physical machine. With a container, I could run hundreds, if not thousands of lightweight instances simultaneously. Each container doesn't need to have its own full-blown operating system, libraries and everything else associated with running a server."

Docker lacks native storage features for data protection such as replication and snapshots.

Docker containers don't natively provide persistent storage

Lack of data persistence and portability is also holding back containers from full-blown use in shared storage, Schulz said.

"You can move containers around, but here's the catch: Where is the data? You need to use a container that's configured for persistent storage," Schulz said.

Newcomer Portworx is trying to tackle that problem. Portworx has designed PWX software for scale-out block storage in Docker volumes, but it will not be available before 2016. Portworx CEO Murli Thirumale said PWX would enable containerized applications and their associated data to be ported between on-premises and cloud storage.

"We're managing the storage by attaching via graph drivers, not as an externally mounted storage," Thirumale said. "We're native to Docker and have the ability to retain whatever file system the Docker container uses. We keep that data while we mount and provide access to a block storage layer below that. The idea is that we're letting that container have access to a block of storage as it's moved from node to node."

PWX is designed to run on x86 servers and Amazon Elastic Compute Cloud Instances, and it will consist of a block storage layer overlaid with orchestration and scheduling of container-level snapshots.

Portworx' debut comes on the heels of a partnership this month by EMC and ClusterHQ. EMC used ClusterHQ's Flocker software to develop pluggable drivers for Docker shared storage on its XtremIO and ScaleIO storage. The Flocker agent lets users move containerized applications and their data across a cluster of servers.

"Moving a storage volume from one container to another on the same server is mildly but not terribly interesting. A lot more valuable is being able to move a container from one physical server to another physical server and have its data go with it," ClusterHQ CEO Mark Davis said.

Container storage is new twist on familiar idea

Crump said companies provisioning storage for Hadoop clusters will notice similarities to Docker.

"Hadoop started with everything being local storage, and over time shared storage has become the preferred way to configure a Hadoop cluster. Docker is much the same way in that most implementations are using local storage. But as your Docker environment starts to scale, you'll want isolated containers that have the ability to communicate with each other," Crump said.

Docker terminology may seem arcane to storage admins, but the concept is similar to creating a logical bucket of storage in Amazon Web Services or firing up database instances for Oracle applications, Schulz said.

"The simplest way for storage administrator to think about Docker is that it's just another application that needs storage," he said. "They need to provision storage for containerized apps, just as they do for the hypervisor, a bare-metal server or database backups."

Next Steps

Enterprises slowly embrace Docker containers

Challenges with Docker containers and data protection

EMC adds Docker storage drivers with containerization on its mind

DockerCon 2016 brings new persistent storage developments

Dig Deeper on Data center storage