Managing and protecting all enterprise data

Nomad_Soul - Fotolia

Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

The hidden cost of SDS products

Though software-defined storage products are praised for their cost efficiency and flexibility, many have overlooked the hidden costs for IT.

Where does the majority of the value reside in a typical enterprise storage array? Is it the software, the hardware or both?

With all the attention and hype around software-defined storage (SDS) lately, the industry seems to be leaning in the direction of software. The noise isn't just coming from small storage startups, either. EMC, HP and IBM, just to name a few, are all offering SDS products.

Despite the confusion around SDS technology, many organizations will likely add SDS products to their list of future storage initiatives, betting on the promises of SDS, all of which look enticing:

  • Hardware flexibility: Separating storage software from hardware allows organizations greater choice in what hardware they deploy and when they deploy it.
  • Access to newer hardware technologies sooner: This flexibility allows for faster or higher-capacity storage to be integrated as soon as the hardware is available, instead of having to wait for a new array to be released four to five years later.
  • Simplified license management: There is no need to buy new or upgrade storage software feature licenses when procuring the next generation of hardware.
  • Support for multiple generations: The ability to integrate multiple hardware revisions over the lifetime of the system. This allows organizations to incrementally upgrade hardware with data in place, eliminating fork-lift upgrades.

However, there is a potentially hidden cost related to SDS that few in the industry are talking about. The mixing and matching of hardware made possible by SDS can shift the cost -- or the risk -- of integrating the software and hardware to the end-user.

While many like to use the term "commodity hardware," in truth, there is no such thing. I spent a portion of my career as a storage engineer, and I still have nightmares of the transition from U160 to U320 SCSI. I remember working long hours in a lab with sets of presumably commodity SCSI hard drives, each creating a unique interaction on the SCSI bus. Drives from vendor A would work fine. Drives from vendor B would work fine. However, if drives from vendor A and B were together, the entire system would break. The results would vary based on controller firmware, drive manufacturers and drive firmware revisions. After months of detailed engineering and test analysis, we would be able to release a qualified and validated system that worked.

When choosing an SDS product, it is critical to evaluate whether the vendor can truly offer the benefits of SDS.

Times and technologies have changed to some extent, and some may argue that drive standards have improved. But I would argue that new hardware technologies, such as solid state, are evolving every day. The bottom line is that if we extend the idea of storage software abstraction to its fullest, it should be able to work with any hardware. If we see that as truly desirable for SDS deployments, the number of possible technology combinations in a system over the life of the software could become endless. In this scenario, the responsibility -- and cost -- to validate and integrate new hardware technologies will fall primarily to IT.

Organizations that are evaluating SDS products today often recognize this challenge and ask for hardware options that have been qualified as a way to help mitigate the risk. Many SDS offerings also have an appliance option to ease the integration concern. However, some might argue that this isn't truly software-defined storage. Additionally, SDS products that target large content repository storage workloads, such as object storage, create multiple copies of data or use erasure coding to improve resiliency, reducing the risk of data loss if a non-validated hardware component is deployed.

I expect some organizations to welcome the opportunity to deploy storage technology as software. These organizations may be large enough to support their own qualification efforts and procure enough hardware where the scale could justify the implementation of non-vendor qualified components. For these deployments, SDS technology as a software-only deployment may make tremendous sense.

Organizations will have to make a choice based on what is best for their business. As noted, SDS players are recognizing this challenge and responding by delivering appliances or providing lists of certified components. When choosing an SDS product, it is critical to evaluate if the vendor can truly offer the benefits of SDS -- whether it is delivered as software or as hardware.

Article 7 of 8

Next Steps

Software-defined storage can ease long-term data retention

SDS vendor buyer's checklist

SDS advantages still to come

Dig Deeper on Software-defined storage

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

How do you rectify the hidden cost of SDS products with the benefits they offer?
I think first, you have to understand what you're potentially NOT getting with SDS.  You mentioned that it can take years for the big vendors to bring new technology to market.  Well, a lot of this is taken up in testing. When you buy a complete system, you're getting the benefits of limits testing, functional testing, failover testing when you inject errors, soak testing to see what happens with systems under sustained load over time, compatibility testing, and significantly, regression testing to ensure that new things don't break old things, because they usually do.  So, if any of these issues are important to you, ask your suppliers for a description of how they test.  Full disclosure:  I'm with a testing vendor, Load DynamiX.  We are currently working with leading SDS software and hardware vendors to quality their joint configurations in performance and limits testing.  Along with the other tests they are doing, you SHOULD wind up with a solution that's got a lot of the benefits of SDS with fewer risks. Shameless plug:  look for our certifications soon.  Or look for other evidence of real testing.
Well, I felt Mr. Sinclair's pain when he mentioned transitioning from SCSI U160 to U320. Back in the day, there were a number of SCSI HBA vendors, and people had their favorites like Adaptec, LSI or Bustek. The best chance of avoiding problems was sticking with a server vendor's list of SCSI HBAs and HDDs they verified as being compatible. Today, a commodity storage server means there isn't much in the way of proprietary hardware and firmware built into the server that would prevent it from being used with any number of software defined storage solutions. In this situation the server hardware is pretty much a non-issue, but it does leave the burden of proof on customers who implement a software-defined storage solution in their environment. Some SDS vendors supply hardware compatibility lists and reference architectures, but as the server hardware and OS software vendors discovered years ago, certifying software on hardware takes time and costs money. A competent IT shop can probably cover this, but customers who want to "stack'em and rack'em" don't want to spend time testing their commodity storage server hardware and SDS software. Plug-and-play appliances, which are usually commodity in nature, and are fully supported by the SDS vendor, is a cleaner and faster approach to getting the job done.
Scott identifies a key issue with SDS. If you are moving to SDS, you need a proper testing process to make sure everything will reliably work "at scale" before you deploy into production. Load Dynamix offers an easy to use testing platform for SDS deployments.

Get More Storage

Access to all of our back issues View All