EMC this week introduced software plug-ins for its XtremIO and ScaleIO storage systems to enable shared storage with portable Docker storage containers.
EMC said the APIs were developed in collaboration with startup ClusterHQ, maker of open source Flocker software for running entire databases in Docker. ClusterHQ's Flocker framework enables applications and their persistent data to be containerized and moved between nodes in a server cluster. The EMC-Flocker integration allows customers to use XtremIO and ScaleIO as persistent storage back ends for Docker-based applications.
EMC said its customers can get the extensible plug-ins as a free download on GitHub. EMC and ClusterHQ have scheduled a joint demonstration of Flocker at the DockerCon conference in San Francisco next week.
Separate APIs have been developed for all-flash XtremIO arrays and ScaleIO server-side SAN storage. Ken Durazzo, EMC vice president of advanced research and development, said the Flocker-based code for Docker is the culmination of two years' of internal development work around open source software.
"We are starting to see an uptick in customers leveraging stateless application containers and using other back-end systems to handle the state for that data. They were asking us what has to happen when data persistence is necessary," Durazzo said.
Docker containers still not ready for enterprise prime time adoption
Containers are considered a lightweight version of virtual machines (VMs). Open source Docker technology enables Linux-based applications and their libraries to be packaged together with a shared operating system. Docker and Microsoft also announced a partnership to support containers in Windows Server 2016 when it is available.
Proponents say containers eliminate concerns about underlying storage dependencies during application development. On the flip side, lack of portability has hindered adoption of containers in primary storage. Docker containers store an application's logic, but not its data.
Flocker enables applications to move along with their data between development, staging and production. This is similar to what VMware vMotion does with VMs.
"We provide a stable way for storage infrastructure to connect to Docker, have a container stop on one server, and restart on another one. And have its data move with it as it bounces around a cluster," said Davis, who sold his last company, Virsto, to VMware in 2013.
The EMC-ClusterHQ partnership is the latest example of the Docker storage containers ecosystem growing far in advance of the technology being ready for widespread enterprise adoption. On the storage side, Docker lacks data protection features such as snapshots and replication, file locking for security and native external storage support (a shortcoming EMC is addressing with Flocker support).
Holger Mueller, senior analyst at Constellation Research Group, said EMC's Flocker driver could help spur adoption of containers in production storage.
"What is missing so far from Docker is the ability to make your data persistent. If you put an enterprise application on a container, and then the container goes away, you lose both the state and lose transactional data," Mueller said.
Docker APIs are part of EMC's open source storage strategy
The ClusterHQ partnership continues a recent string of EMC open source initiatives. At EMC World 2015 in May, the vendor said it would open its proprietary ViPR storage controller software to community development and unveiled Project Caspian, a forthcoming storage appliance running on OpenStack.
"It's pretty clear that we are moving in a bunch of open source directions, and Docker is another instantiation of that," EMC's Durazzo said.
Mueller said he expects EMC competitors will follow suit and begin writing Docker plug-ins to work with their storage.
"Historically, EMC was usually one of the last to adopt something new. Now that EMC supports Flocker, storage vendors that have been sleeping at the wheel may have to wake up and take notice," Mueller said.
Why Docker storage containers are becoming more popular
The low-down on storage and Docker containers
What the adoption of Docker means for the virtualization world