In this fourth tip of a seven-part video series, Jon Toigo explains how vendors have changed traditional storage hierarchies. Jon, who is chairman of the Data Management Institute and CEO of Toigo Partners International, notes that this is just another factor contributing to the 'storage infrastruggle,' his term describing IT workers efforts to deal with data storage technology. Watch the video above or read the text below for his views on the rightful place of RAM, hard disk, tape and optical storage in the data center.
Please read Toigo's entire video-tip series on data management issues
Data management issues: Updating your strategy to cope with exponential growth
Toigo: Storage admins underestimate virtualization requirements
'Disk everywhere' methodology has a negative impact on long-term data storage
Why capacity usage is more important than capacity allocation when dealing with storage software
Availability, cost and security are just some of the problems with cloud storage
IT decision-making process needs a makeover when it comes to storage technology issues
In my tip regarding the impact of "disk everywhere" dogma on the storage infrastruggle, I noted that cost-per-gigabyte efficiencies enjoyed at the disk drive level aren't being passed to the consumers of finished array products. While a gigabyte of disk drive space may cost 50% less year over year, given falling disk drive prices, array prices tend to accelerate at a rate of approximately 120% annually. This is despite arrays being, for all intents and purposes, merely collections of commodity hard disks.
Actually, the final part of the assertion above is a bit deceptive. There's more to an array than commodity disk drives in a commodity tray inserted into a commodity cabinet. Enterprise arrays typically feature value-add software operated from the equivalent of a server called a controller in array parlance.
There's nothing new about this configuration. There's always been a hierarchy of disk arrays within the broader "storage hierarchy" -- traditionally random access memory (RAM), hard disk arrays, and tape library or optical disc storage -- designed to meet performance and capacity requirements for disk storage. High-performance storage arrays were very expensive cobbles of high-speed, low-capacity disk drives designed to facilitate the highest speed reads and writes of the most demanding transactional applications or databases. For long-term hosting of less frequently accessed data, there were other storage rigs providing high-capacity, slower performing drives. These also tended to be less expensive than their high-performance cousins.
When data needed to be backed up or committed to a deep historical archive, traditionally it was migrated off these two disk array types into a third tier of storage: tape or optical. These media types were considered much slower responders to access requests, but their capacities and low cost made up for any inconveniences in terms of access speed.
A phenomenon we've witnessed over the past decade or so is the deliberate effort of disk array vendors to displace the magnetic tape and optical disk markets by introducing products that co-opt the value case for tier-three technologies. We've seen a blurring of the lines between performance and capacity disk with the introduction of array products that cobble together ranks of both high-performance/low-capacity and high-capacity/low-performance disk into the same rig. To this kit is added auto-tiering value-add software designed to monitor data access frequency on high-performance tiers and migrate less frequently accessed data to high-capacity disk. Such a product combines the functionality of tier-one and tier-two storage arrays according to vendors, eliminating the need for the tier-two product altogether (while also enabling a significant markup of the different drives used in the auto-tiering rig).
Other disk arrays have been offered to the market that bridge tier two (capacity-oriented disk) and tier three (the domain tape and optical previously occupied). These systems tend to feature large complements of high-capacity/low-performance disk drives controlled by value-add software that features functionality such as data deduplication or compression. Vendors tout such capacity-enhancement software to increase the value of each and every disk drive, enabling it to store many times the bits as raw drives, albeit in a compressed or deduplicated state. However, the software used to accomplish data reduction has the effect of marking up the price of each disk drive by 80 to 100 times.
All told, the value-add functionality used to redefine the traditional data storage hierarchy has significantly increased the cost of disk storage, the power consumption of storage in the data center, and the complexity and difficulty of infrastructure management, given that "value-add" software often obfuscates storage resource management tools.
As compelling as a disk-only storage infrastructure might sound, it's one of the most expensive ways to build storage.