carloscastilla - Fotolia
Small World Big Data
Published: 06 Jan 2016
It's hard to make stunning predictions on the future of data storage that are certain to come true, but it's that time of year and I'm going to step out on that limb again. I'll review my predictions from last year as I go -- after all, how much can you trust me if I'm not on target year after year? (Yikes!)
Last year, I said the total data storage market would stay flat despite big growth in unstructured data. I'd have to say that seems to be true, if not actually dropping. Despite lots of new entrants in the market, the average vendor margin in storage is narrowing with software-defined variants showing up everywhere, open-source alternatives nibbling at the edges, commodity-based appliances becoming the rule, and ever-cheaper “usable” flash products improving performance and density at the same time.
Another look at all-flash arrays vs. hybrids
I said all-flash arrays would start beating out hybrids for tier 1 workloads. And, in fact, as flash gets cheaper and denser and folks weigh the Opex of maintaining mixed-storage environments, we've even seen all-flash arrays in use for tier 2 workloads. Taneja Group research conducted in 2015 showed enterprises have big plans to migrate to flash across the board. But, I didn't call out what now seems obvious -- that newly engineered hybrid arrays would be designed for all-flash performance and still offer intelligent auto-tiering to lesser media to optimize capacity and cost. I think we'll need to redraw category definitions this year to help clarify if an array is an all-flash or full flash-engineered hybrid, as opposed to just flash-capable (i.e., you can stick in some SSDs).
The reason I might have overlooked this a year ago might have stemmed from something quite silly -- many vendors have since told us they had to create an all-flash SKU and ship hybrid-capable arrays with all-flash in order to be counted as performance tier 1 flash by some analyst firms. So, even though they could combine comparable performance with some cost-agility tiering, they wouldn't be presented and compared side-by-side with all-flash products. That's just ridiculous. Valid storage comparisons ought to be based on actual capabilities, performance and cost, not SKUs. All-flash products might generally still hold an edge on consistency and Opex and hybrids might kick back on Capex and agility, but let's at least make honest comparisons.
Server-side flash vs. shared storage
I predicted that the future of data storage would see more practical server-side flash products including read- and write-caching over shared storage. That happened and vendors are now also including a server memory tier in their schemes, as if flash didn't provide enough I/O boost. In 2016, I predict that we'll see more “amorphous” offerings in which memory, local flash, intelligent adapters and remote scale-out storage all transparently work together as a single storage system, self-configuring and optimizing based on application QoS and capacity requirements.
How many storage products are too many?
I hinted that there were too many storage products in the market without enough differentiation. This hasn't resulted yet in many vendor consolidation moves, although I'd count EMC and Dell as a big one in progress. Also, HPE seemed to throw down a gauntlet by creating a new enterprise-focused organization with complete end-to-end products and services. This year, I wouldn't be surprised to see some additional big mergers and acquisitions, or as in HPE's case what I might call “enterprise clarification.” NetApp seems vulnerable with everyone aiming at their core NAS business, while IBM is on a warpath to energize its enterprise products. And even though (or maybe because) VCs seem desperate to find and fund potential unicorns, we should see some smaller players start to run out of runway as we move into the future of data storage.
Data protection joins storage
I predicted that we would see data protection features getting baked into storage directly, and we've seen some interesting examples in this area from HPE (between 3PAR, StoreOnce, and Data Protector) and Oracle (e.g., Zero Data Loss Recovery Appliance). At the same time, integrated, auto-tiered cloud back-ends are creeping out, but not heavily marketed yet as most vendors really don't want to cede business to the likes of Amazon. Interestingly, IBM and now Dell/EMC are heavily into their own cloud services, but HPE switched gears on that front.
Application-centric storage taking over
One of the bigger trends I predicted in the future of data storage is well underway as we mark the broader move toward storage that is increasingly application-centric, driven more directly by application needs than merely consumed as generic block or file services. For example, every vendor is gearing up its VMware Virtual Volumes support that enables provisioning, operations and quality of service per virtual machine. Going further, we are starting to see new data-aware storage offerings that use extra metadata over block and file services to optimize performance/capacity, track usage, support infinite versioning or snapshots, and enhance security. Here is an easy prediction -- arrays will continue to get smarter!
I predicted a great future for software-defined products and the hyperconverged appliances they enable. But I will admit that the phrase “software-defined” has lost much of its importance as every marketing genius reasons that their product includes software, so it must also be software-defined. Still, our research shows that more than 30% of enterprise respondents are envisioning hyperconvergence as their future data center architecture. However, it won't be long before those same marketing geniuses figure that their product also uses compute, memory, and networking, so it must also be ultra-super hyperconverged.
Finally, the obvious prediction I made about the growing success of private cloud storage is holding true. Some of the top-selling storage products in 2015 are object storage-based, benefitting greatly from private cloud-building initiatives. In addition to inexorable Web-friendly application development, everyone also wants a corporate file sync-and-share product, and many are moving up into corporate big data lakes.
Forecast looks bright for Opex
OK, so here is a new prediction on the future of data storage for 2016. The Opex of storage -- including provisioning, troubleshooting, maintenance, upholding availability and performance SLAs, migration/transitions, ensuring security or compliance, and so on -- will become a more important investment consideration even among senior business and financial officers who traditionally only trust Capex spreadsheets.
Here's why: Storage media prices keep dropping and capacity efficiencies continue to evolve (e.g., inline dedupe/compression). And IT is better tracking and exposing total storage costs as they build their own clouds to compete with outside alternatives. The push to cloud computing has helped financial-minded folks get used to comparing ongoing cost structures instead of focusing on one-time investments. Also, as IT evaluates new storage products, they are recognizing that the only way to do more while running lean and mean is to demand increasing end-to-end automation and built-in intelligence.
What's ahead for the data storage industry?
A look back at one set of data storage predictions
Sanbolic CEO discusses the future of flash storage