Hard disk drive technology trends

Solid state is emerging as a viable enterprise storage alternative, but there's lots of life left in hard disk drives, with higher capacity, greener and more capable drives coming.

Although solid state is emerging as a viable enterprise storage alternative, there's still plenty of life left...

in hard disk drive technology, with higher capacity, greener and more capable drives on the way.

Nobody is seriously suggesting the era of hard disk drive (HDD) technology is about to end. To the contrary, HDD vendors continue advancing the technology.

Still, a few doubts arise as solid-state drives (SSDs) and other solid-state technologies take on selective enterprise storage chores. "The amount of SSD replacing hard disk drives at present is miniscule," said Mark Geenen, chairman, International Disk Drive Equipment and Materials Association (IDEMA). Yet, IDEMA began accepting SSD vendors into its ranks in 2009.

More telling, maybe, are the enterprise storage managers willing to experiment with solid-state storage alongside HDD. "We're testing a couple of Fusion-io cards in some servers for a couple of high-performance trading applications," said Kevin Fiore, CIO at San Francisco-based investment firm Thomas Weisel Partners LLC. The company wants to determine if the 160 GB and 320 GB solid-state cards make enough of a difference to justify the extra thousands of dollars they cost.

Solid-state storage, however, will remain a small piece of the enterprise storage picture for some time (see "Is solid-state storage in your future?" below). Instead, what's driving enterprise storage is the continuation of HDD trends that have been underway for six years or more. The trends include the steadily falling cost per gigabyte of HDDs, the shift from 3.5-inch to 2.5-inch form factors, and a movement away from 15K rpm Fibre Channel (FC) to 10K and 7,200 rpm SAS and SATA drives. And don't forget ever-increasing disk densities reflected in the emergence of 1 TB and 2 TB drives, with 3 TB and 4 TB capacities on the horizon.

Is solid-state storage in your future?

The cost of solid-state storage fell dramatically in 2009. Prices will dip at a slower pace in 2010 but pick up again after that, according to Jim Handy, solid-state drive (SSD) analyst at Objective Analysis in Los Gatos, Calif. Although solid-state storage comes in a variety of forms, NAND flash has emerged as the leading choice for enterprise storage.

Until recently, solid state was too expensive for anything but situations demanding the highest high performance. In 2009, SSD cost, on average, 20 times more than Fibre Channel hard disk drives (FC HDDs) on a cost/GB basis, Handy noted. But if you compared them based on cost/IOPS, SSD blows away HDD.

To Mark Teter, chief technology officer at Advanced Systems Group, SSD's low cost/IOPS spells doom for FC HDDs. He advises companies requiring high-performing storage to replace their large numbers of FC spindles with a much smaller amount of SSD.

Already storage vendors and drive makers alike are embracing the HDD/SSD combination. EMC Corp., for example, has incorporated SSD as what amounts to tier 0 in its enterprise storage hierarchy and recommends its FAST product to automatically move data between SSD and HDD tiers. Hewlett-Packard Co., IBM, Oracle-Sun and many other storage vendors all offer SSD storage options.

Steadily evolving HDD technology will also force storage managers to revisit issues like encryption, compression and data deduplication. IT managers can now decide the best place to do encryption or deduplication: on the server, the array, an appliance or the disk drive. With the newest generation of HDDs, all options are on the table.

Similarly, new HDD technology offers more energy-saving options, primarily via disk spin-down technologies. But the massive amounts of capacity on a single drive also raise questions about the efficacy of RAID 5 as a data protection technique (see "The end of RAID," below).

The end of RAID

For industry analyst Mike Karp, the growing capacity of multiple terabyte disk drives inevitably increases disk errors. "For every unit of capacity we know there will be a certain number of errors," he said.

Organizations traditionally relied on RAID 5 to overcome those errors through the use of parity data. However, because of the time it takes a system to rebuild a 1 TB drive using RAID 5 the likelihood that another drive may fail increases, compounding the problems. Karp's solution is object-based RAID, which relies on small units of storage and allows the system to rebuild the drive in smaller increments. Others suggest RAID variations that use multiple parity disks, such as RAID 6, can do the job. (See "Alternatives to RAID")

Price/Capacity curve

The data storage industry has enjoyed a long ride up the price/capacity curve. Each year vendors pack more data onto disks with a corresponding drop in the cost per gigabyte. It's the storage industry's version of Moore's Law.

In recent years, 40% annual price/capacity improvements have been typical, but that pleasant pace may be slowing. "It will be increasingly difficult to get these kinds of gains," said Mark Nossokoff, a senior member of the strategic planning team at LSI Corp.'s Engenio Storage Group. The latest gains resulted from the vertical stacking of bits on the disk to overcome the limits of the superparamagnetic effect. Future increases will need a new technology advance. Bit-patterned media (BPM), heat-assisted magnetic recording (HAMR) and microwave-assisted magnetic recording (MAMR) are the leading contenders to replace perpendicular technology (see "New technologies pack more data on disks," below).

New technologies pack more data on disks

Several technologies are vying to produce the next breakthrough in packing more bits on a piece of storage media. Currently, perpendicular magnetic recording (PMR), which stacks bits vertically on the surface rather than laid out horizontally, is the technology of choice, but it will hit the superparamagnetic limit within a few years. But alternative technologies are emerging, including:

  • Bit-patterned media (BPM) stores each bit as a nanometer-scale pattern of grains on the media. As described by Hitachi, it creates an ordered array of highly uniform islands, each island capable of storing an individual bit.
  • Heat-assisted magnetic recording (HAMR) uses heat to stabilize the tiny stored bits, allowing smaller bits to be recorded. HAMR, however, creates heat, which runs counter to the industry's growing green storage impulses.
  • Microwave-assisted magnetic recording (MAMR) writes bits at different layers of the media.

HAMR will likely be ready for production by 2013, with MAMR probably ready a year later, according to Mark Nossokoff, a senior member of the strategic planning team at LSI Corp.'s Engenio Storage Group.

How much capacity can be squeezed onto a 3.5-inch platter using current technologies? "I expect 8 TB within four to five years," predicts Greg Schulz, founder and senior analyst at StorageIO Group in Stillwater, Minn. With new materials and different technologies, Schulz won't be surprised to see HDD capacity continue to increase dramatically. Jon Toigo, CEO and managing principal at Toigo Partners International, reports a Toshiba breakthrough that stores 4 TB per square inch (not per 3.5-inch platter). Toigo predicts a 4 TB/sq. in. disk drive within 36 months. Similar advances and new materials promise equally dramatic increases in tape capacities.

A general cost per gigabyte of HDD storage is difficult to pin down due to the wide variety of HDDs with different capacities and speeds, and the different ways HDDs are packaged and sold. Still, there are ballpark approximations. For example, a high-performance FC drive in an array would run approximately $80/GB; stick it in a SAN and the cost bumps up to $180/GB, according to Toigo. Figure approximately $60/GB for a SAS drive in an array and $160/GB in a SAN; SATA drives are about $40/GB.

Prices are rock-bottom for consumer-quality hard disk drives with some retailers selling 1 TB 5,400 rpm drives with a USB connection for $89 or less -- about 9 cents per gigabyte! These may not be industrial-strength drives, but at that price a manager could buy a handful and back up an entire workgroup or department several times over in case one failed. Enterprise tape is a bargain too, Toigo added, with prices at about 44 cents/GB.

Given the low cost/GB, especially for drives with capacities of 1 TB or more, some storage managers are snapping up the largest drives they can buy. "We're always buying more hard drives. We buy SATA for the low end, SCSI for the midtier and we even have a tiny amount of SSD," said Ben Higginbotham, director of new technology at WhereToLive.com, a Web development firm and SaaS provider for the real estate industry. Most recently the company has been buying 1 TB SATA drives in its ongoing quest for more storage capacity. Next up for the company as it continues to ride the price/capacity curve are 2 TB drives.

Thomas Weisel's Fiore is also constantly seeking greater capacity with both 3.5-inch and 2.5-inch HDDs. "We want to get the most capacity in the smallest footprint. It's a real estate issue with us" he said. Going from 1 TB to 2 TB doubles the amount of capacity for each drive shelf. Similarly, with 2.5-inch HDDs he can pack more drives into the same space.

But not every storage manager is enamored with the biggest drives. "The cost per gigabyte is appealing, but the 1 TB and 2 TB drives also scare me because of the increased risk of failure," said Darrell Stymiest, director of IT operations at UGL Unicco, a facilities services company based in Newton, Mass. Instead, Stymiest prefers to buy smaller drives, currently 300 GB or 500 GB capacities.

Form-factor and interface changes

The greatest capacities are still in the 3.5-inch HDD form-factor, although there's a gradual transition in enterprise storage to the 2.5-inch HDD, which currently tops out at approximately 600 GB. "The 3.5-inch disk is still a major part of the market," said Teresa Worth, senior marketing product manager at Seagate. "By 2012, 2.5-inch disk will pull even; from then on, 2.5 inch will take over." IDEMA's Geenen concurred: "Enterprise storage today is still 3.5-inch disk, but that's changing to 2.5 inch."

The appeal of the 2.5-inch hard drive technology is energy efficiency. "The 2.5-inch drive offers a 60% improvement in power," Worth said. With vendors like Seagate adding intelligence to the devices that allows storage managers to selectively reduce disk spinning, hard disk drive power consumption can be reduced even more.

Lincoln, Neb.-based trade magazine publisher Sandhills Publishing Co. standardized on 2.5-inch HDDs a year ago. "We do a lot with SQL Server and for that we want a lot of fast-spinning spindles," said Kim Mehring, IT manager. The 2.5-inch form factor enables Sandhills to pack more disks in each array, which means more spindles. By aggregating more spindles, Mehring can boost I/O performance. He considered SSD for its performance, but the cost was still too high for his budget.

Along with the form-factor transition, the industry is witnessing an interface transition from FC for tier 1 enterprise storage to SAS, and to SATA for tier 2 and archival storage. "With 2.5-inch drives we're seeing a shift to SAS for performance and SATA for high capacity," StorageIO Group's Schulz noted.

But FC drives are far from dead. EMC Corp., for example, recently increased its standard 15K FC drive from 300 GB to 600 GB, according to Scott Delandy, senior product manager for infrastructure products at EMC. Where high-performance storage is required, however, EMC now recommends a combination of SSD and SAS or SATA. "You can replace 146 GB 15K drives with flash and get 30 times greater IOPS for the money," he said. That kind of disk strategy requires automated storage tiering.

If the flash tests work out well at Thomas Weisel Partners, Fiore can imagine shifting away from FC over time to a combination of flash and SATA. "With FC, the largest capacity we can get is 400 GB. We're at the mercy of the array vendor," he said. Given his concerns about storage footprint, Fiore has been steadily buying more SATA than FC disk.

Encryption and going green

Chores previously handled by servers or appliances, such as encryption and data deduplication, can now be performed directly on hard disk drives. Specifically, vendors are introducing HDDs with built-in encryption. "Encryption really belongs at the drive level," said Seagate's Worth. The company already offers a self-encrypting drive and expects it to become the standard choice as part of the normal enterprise HDD upgrade process.

Storage managers, however, may not be in any hurry to buy built-in HDD encryption. "We don't worry about encryption, and we do our de-duplication on our Compellent SAN," WhereToLive.com's Higginbotham said. IDEMA's Geenen sees encryption as a way for HDD vendors to add something they might charge more for.

"We encrypt laptops that leave our premises or specific data," Thomas Weisel's Fiore said. "We see no need to encrypt entire drives on the SAN."

Green storage takes several forms with HDD. First, larger capacities mean companies can get the same amount of storage while spinning fewer drives, which saves energy. Second, "vendors are building intelligent power management into the drives that allows them to drop the rpm speed or turn off writes," StorageIO Group's Schulz said.

Storage managers are interested in green disk drive technologies, but they're not exactly jumping at the products. "We don't spin down disks, but anything that saves power, like using large, slower drives, we do," WhereToLive.com's Higginbotham said.

Hard disk drive technology is undergoing significant shifts in form factor, capacity, interfaces and capabilities. It's not, however, going away -- certainly not in the next decade or maybe ever. Solid-state storage will get cheaper and work its way in increasingly greater volume into the enterprise storage strategy, but more likely as a complement rather than a replacement for hard drives.

BIO Alan Radding is a freelance writer and frequent contributor to Storage magazine and SearchStorage.com.

Dig Deeper on Data storage management