Toigo: A brief history of storage management systems

Toigo: A brief history of storage management systems

Toigo: A brief history of storage management systems

Date: Jan 18, 2013

In this Storage Decisions 2012 video, Data Management Institute chairman Jon Toigo gives a brief history of data storage management systems. Watch the video above or read a portion of the transcript below to learn Toigo's take on how storage evolved, and what worked and didn't along the way. In Toigo's world, storage has finally evolved to a place that it can be considered a service.

We used to set up data classes, storage classes and management classes. I'm not saying it was the only way to do it, but it was all handled through system-managed storage and hierarchical storage management on the mainframe. It was handled at the server, and basically we had a topology for storage that would migrate the data from the most expensive and fastest storage down to the least expensive, slowest and highest capacity storage as a routine. That's a great solution to the problem. It has flaws -- it was hard to change it once the status of certain types of data changed or new data was added; it was not a very flexible system, but the fact was that we had tools that could work within this schema.

What we came up with [next] was this: islands of storage. Every single box out there has its own set of functionality, and each one is managed by its own little CD that comes with the box. It's fine as long as we only have one box. When we get to 10 boxes or 100 boxes or 1000 boxes, it becomes rather unwieldy. It's like surfing the internet -- we're going to go from site to site to site.

We came up with solutions like storage resource management (SRM) that nobody bought. A lot of companies have gone bankrupt trying to solve storage resource management, which cobbles together all of these products. The SRM guys will tell you a different story. They'll say, 'Do you know how difficult it was to create an SRM product that would talk to the APIs of every single vendor's product?' It was like they weren't interested in being managed in common. Because if you can manage storage in common, you can unplug an EMC rig and plug in an HDS rig. People will begin to realize that there isn't that much of a difference between any of this equipment. The brand name doesn't count.

The next generation -- this happened in the '90s -- we decided to create something called a 'storage area network.' SANs are networks that aren't really networks. They're physical layer interconnects using serial iSCSI. There's nothing wrong with that principle, except that you have no management layer in a SAN. It's not truly a network. A network protocol has a management layer, and there isn't one in a SAN.

So, the switch vendor provided us with a switch. It was made to the exact standards and specifications of NTC 11 with absolutely certainty that it wouldn't work with a competitor's switch, which is also completely standards-adherent. That's what happens with switches or any product that's built on specifications that are articulated by vendors. We used SRM, and what we discovered was that now we just have one more thing to have them monitor, and that was the switch layer.

Today, we confront an interesting turn -- VMware, the 'V party.' Basically, what they're saying is, 'We want to go back in time to this mythical time that never existed. We want to be the next mainframe. We want to be the mainframe "mini me." Basically, we're going to add a storage hypervisor to the VMware stack.' That happens this year, maybe early next year. And you think, 'Well that sounds like a good idea because the same thing that's virtualizing all of our workload can also be used to virtualize all of our storage, and we can assign storage resources to manage specific workloads.' It sounds a lot like the mainframe days.

Of course the right way to go is to stream ahead and look for something that's based on an open standard like REST to manage all your gear. I currently use REST management with X-IO gear, and I can manage 96 PB of storage with my smartphone. REST is four primitives: get, delete, save and something else; I forget. Anyway, we could be building on that. It's already been demonstrated that we can do it. There's a site called '' where you can see the stack of software that's required to do it. It takes about a week to implement. Hardware vendors refuse to do it.

Now it sounds like a good idea to go this route that VMware's pitching, unless you have an environment where you not only have VMware, but you have some Hyper-V and you have some Citrix and you have some physical servers. Then, you're going to have to start segregating your storage infrastructure. All the VMware stuff has to be segregated in that SAN. You're going to have to aggregate -- which is what we've done for the last 10 years, creating Fibre Channel fabrics. Then segregate, then aggregate. Will it solve the problem? No. It makes it worse. It makes it more complex.

What we need is a hardware and software agnostic hypervisor. You drop it right in place with all the storage you have. You don't have to change the cabling, the storage, the switch infrastructure -- nothing. You virtualize it all. You create virtual volumes -- pools of storage. You can do thin provisioning across all rigs, not just one. Continuous data protection across all rigs, not just one; tiering on all rigs, not just one. Mirroring on all, replication on all, snapshots on all, virtualization on all. The advantages include things like a twofold performance increase on the underlying storage because all of the writes are acknowledged at the DRAM cache, and not waiting for the storage to do the write operation and then respond. You can manage capacity and performance in a unified way, plus data protection. If you surface all of those functions, you can assign them easily to each workload. And it doesn't care whether that workload is coming from VMware, Citrix or Hyper-V.

Storage is finally a service. We've found a way to develop it. It beats the socks off of SRM.

More on SAN management

  • canderson

    Schulz: EMC software-defined storage strategy has muscle behind it

    VIDEO - Industry expert Greg Schulz discusses EMC's focus on software-defined storage at its annual user show.
  • canderson

    DataCore Software's take on software-defined storage

    VIDEO - In this video, DataCore Software Chairman Ziya Aral talks about the true meaning of software-defined storage.
  • canderson

    Making the switch to VHDX

    VIDEO - Brien Posey explains the benefits of using the VHDX format and provides step-by-step video instructions on how to convert VHD files to VHDX files.
  • Cisco expands FC, converged Ethernet storage switching

    News - A new 96-port FC switch and greater 40 gigabit converged Ethernet support for Nexus switches expands Cisco's support for all storage protocols.

    ( May 21, 2015 )

  • SCSI (Small Computer System Interface)

    Definition - Definition: The Small Computer System Interface (SCSI) is a set of parallel interface standards used to attach disk drives and other peripherals to a computer.
  • storage area network (SAN)

    Definition - Definition: A storage-area network (SAN) is used to move data between servers and the various storage devices those servers use.
  • software-defined storage

    Definition - Software-defined storage (SDS) is an approach to data storage in which the programming that controls storage-related tasks is decoupled from the physical storage hardware. Software-defined storage is part of a larger industry trend that includes software-defined networking (SDN) and software-defined data centers (SDDC).
  • Tucci: EMC Federation provides fastest route to cloud

    News - Joe Tucci and other top EMC execs argue at EMC World 2015 that all of the Federation pieces make the best on-ramp to private and hybrid clouds.

    ( May 06, 2015 )

There are Comments. Add yours.

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: