The Next Era of Data Storage Will Be One of Massive Scale

by Larry Freeman, NetApp, Senior Technologist

Free Download

Cloud Management In A Hybrid Cloud World

Learn More

The evolution of data storage has been defined by three distinct eras:

  • Mainframes and disk drives
  • Client/server and networked storage
  • The Internet and storage virtualization

New technology and rapid data growth have been the drivers of each transition from one era to the next. And now we are on the verge of a new era of data growth, this one driven by mobile technologies and machine-generated data.

This next era will require storage architectures capable of 24/7 operation and massive scale in order to keep up with a veritable tsunami of new data.

To prepare for the scope of the changes that will be required in the not-too-distant future, it helps to look back at the previous eras and the impact each has had on data storage technology.

First Era: Mainframe and Disk Drives
The first era began in the 1950s, as mainframe computers gave innovative businesses an edge on the competition. Business operations could be done faster and more accurately than ever before. Time compression and business accuracy were the major attributes these early computers brought (and still bring today). However, storing data on punch cards or as hard copy on green bar paper quickly proved ineffective. Efforts to find better ways to retain growing volumes of data led to the innovation of a new-fangled storage device: the disk drive.

Second Era: Client/Server and Networked Storage
The second era was driven by the move to client/server computing, which delivered new capabilities to rank-and-file workers. During this era, business productivity and user-generated data grew by leaps and bounds, fueled by software tools from Microsoft, Oracle and many others. Breakthroughs in storage technologies were needed and were delivered by way of several innovations, such as small form-factor disk drives configured into storage “arrays” that communicated across a common interface (SCSI) and included critical RAID device protection.

Over time, these independent arrays were networked into SAN and NAS storage networks to further accelerate the movement of large volumes of data between clients and servers. These innovations set the stage for an even greater proliferation of data as we entered the 21st century.

IDC Whitepaper – Discover the Benefits of Clustered Storage

Third Era: The Internet and Storage Virtualization
The third (and still current) era of data storage was a direct result of the opportunities brought by the Internet. Email, e-commerce and video streaming were just a few of the Internet-enabled workloads that marked an era of relentless data growth. But while data was again creating new business opportunities, it was also creating misery for the IT architects responsible for storing the data. Storage optimization became critical to success and is being delivered in the form of storage virtualization, storage management tools and cloud storage.

The Next Era: The Internet-of-Things and Massive Scale-out Storage
Today, mobile technologies and machine-generated data are accelerating the rate of data growth and driving a transition into the next era of data storage. According to Cisco CEO John Chambers, “the Internet of Things” will soon reach 50 billion devices, and it is predicted that it will have five to 10 times the impact on society compared with the Internet alone.

This next era of data storage, like the others before it, will require new technologies.  While many innovations will be required, one thing is certain: Scale-out storage architectures will be foundational to success. Scale-out storage is the ability to create utility-like storage pools from multiple networked storage devices, and it is rapidly becoming a “must-have” capability for enterprise IT shops. Just as in earlier days when punch cards proved an ineffective means to store data, many of today's networked storage arrays will prove ineffective as data scales faster than legacy networks and traditional scale-up storage arrays can accommodate.

Scale-out storage offers a promise of vast data storage pools that operate with little human intervention and perform consistently under varying loads, just as an electrical power grid or a public water distribution system does. Scale-out storage long ago passed the experimental phase and is now a reality. Hyperscale cloud providers such as Amazon S3, Google and Microsoft Azure all operate non-disruptive, infinitely scalable storage architectures at exabyte scale. As scale-out technologies become more pervasive, they will become widely available from cloud service providers, Software as a Service companies and traditional storage array vendors.

NetApp Scale-out Storage
NetApp is a leading innovator of scale-out storage technology. Using Data ONTAP, the world’s most popular storage operating system, storage systems can be clustered into a single storage pool of more than 50 petabytes (PB), can configure themselves with minimal human intervention via a storage automation engine, and operate completely non-disruptively—even during component upgrades and data migration events.

At the current rate of growth, the typical enterprise IT shop will surpass 1PB of online data by the year 2018, although many of our customers have already far surpassed this milestone. The coming era of data storage will undoubtedly place new stress on IT departments as they accept massive data growth as the new normal. Scale-out storage will be the first step in relieving their pain.

A frequent speaker and author, Freeman’s current role at NetApp is educating IT professionals on the latest trends, techniques, and best practices in data storage technology. He authored the book Evolution of the Storage Brain and hosts the popular blog “About Data Storage.”


No portions of this document may be reproduced without prior written consent of NetApp, Inc. Specifications are subject to change without notice. NetApp, the NetApp logo and Go further, faster, are trademarks or registered trademarks of NetApp, Inc. in the United States and/or other countries. Click here for a full listing of NetApp trademarks. All other brands or products are trademarks or registered trademarks of their respective holders and should be treated as such.

Show more