In this Storage Decisions 2012 video, Data Management Institute chairman Jon Toigo gives a brief history of data storage management systems. Watch the video above or read a portion of the transcript below to learn Toigo's take on how storage evolved, and what worked and didn't along the way. In Toigo's world, storage has finally evolved to a place that it can be considered a service.
We used to set up data classes, storage classes and management classes. I'm not saying it was the only way to do it, but it was all handled through system-managed storage and hierarchical storage management on the mainframe. It was handled at the server, and basically we had a topology for storage that would migrate the data from the most expensive and fastest storage down to the least expensive, slowest and highest capacity storage as a routine. That's a great solution to the problem. It has flaws -- it was hard to change it once the status of certain types of data changed or new data was added; it was not a very flexible system, but the fact was that we had tools that could work within this schema.
What we came up with [next] was this: islands of storage. Every single box out there has its own set of functionality, and each one is managed by its own little CD that comes with the box. It's fine as long as we only have one box. When we get to 10 boxes or 100 boxes or 1000 boxes, it becomes rather unwieldy. It's like surfing the internet -- we're going to go from site to site to site.
We came up with solutions like storage resource management (SRM) that nobody bought. A lot of companies have gone bankrupt trying to solve storage resource management, which cobbles together all of these products. The SRM guys will tell you a different story. They'll say, 'Do you know how difficult it was to create an SRM product that would talk to the APIs of every single vendor's product?' It was like they weren't interested in being managed in common. Because if you can manage storage in common, you can unplug an EMC rig and plug in an HDS rig. People will begin to realize that there isn't that much of a difference between any of this equipment. The brand name doesn't count.
The next generation -- this happened in the '90s -- we decided to create something called a 'storage area network.' SANs are networks that aren't really networks. They're physical layer interconnects using serial iSCSI. There's nothing wrong with that principle, except that you have no management layer in a SAN. It's not truly a network. A network protocol has a management layer, and there isn't one in a SAN.
So, the switch vendor provided us with a switch. It was made to the exact standards and specifications of NTC 11 with absolutely certainty that it wouldn't work with a competitor's switch, which is also completely standards-adherent. That's what happens with switches or any product that's built on specifications that are articulated by vendors. We used SRM, and what we discovered was that now we just have one more thing to have them monitor, and that was the switch layer.
Today, we confront an interesting turn -- VMware, the 'V party.' Basically, what they're saying is, 'We want to go back in time to this mythical time that never existed. We want to be the next mainframe. We want to be the mainframe "mini me." Basically, we're going to add a storage hypervisor to the VMware stack.' That happens this year, maybe early next year. And you think, 'Well that sounds like a good idea because the same thing that's virtualizing all of our workload can also be used to virtualize all of our storage, and we can assign storage resources to manage specific workloads.' It sounds a lot like the mainframe days.
Of course the right way to go is to stream ahead and look for something that's based on an open standard like REST to manage all your gear. I currently use REST management with X-IO gear, and I can manage 96 PB of storage with my smartphone. REST is four primitives: get, delete, save and something else; I forget. Anyway, we could be building on that. It's already been demonstrated that we can do it. There's a site called 'CortexDeveloper.com' where you can see the stack of software that's required to do it. It takes about a week to implement. Hardware vendors refuse to do it.
Now it sounds like a good idea to go this route that VMware's pitching, unless you have an environment where you not only have VMware, but you have some Hyper-V and you have some Citrix and you have some physical servers. Then, you're going to have to start segregating your storage infrastructure. All the VMware stuff has to be segregated in that SAN. You're going to have to aggregate -- which is what we've done for the last 10 years, creating Fibre Channel fabrics. Then segregate, then aggregate. Will it solve the problem? No. It makes it worse. It makes it more complex.
What we need is a hardware and software agnostic hypervisor. You drop it right in place with all the storage you have. You don't have to change the cabling, the storage, the switch infrastructure -- nothing. You virtualize it all. You create virtual volumes -- pools of storage. You can do thin provisioning across all rigs, not just one. Continuous data protection across all rigs, not just one; tiering on all rigs, not just one. Mirroring on all, replication on all, snapshots on all, virtualization on all. The advantages include things like a twofold performance increase on the underlying storage because all of the writes are acknowledged at the DRAM cache, and not waiting for the storage to do the write operation and then respond. You can manage capacity and performance in a unified way, plus data protection. If you surface all of those functions, you can assign them easily to each workload. And it doesn't care whether that workload is coming from VMware, Citrix or Hyper-V.
Storage is finally a service. We've found a way to develop it. It beats the socks off of SRM.