Manage Learn to apply best practices and optimize your operations.

A brief history of storage management

An amusing look at the history of storage with Jon William Toigo.

Since the Dawn of Man, nearly everyone has agreed that data storage management is tough work. Early efforts to manage storage focused on the medium itself.

  • Paleolithic Man thought he had the problem of data permanency and non-repudiation licked by painting information on cave walls. However, he discovered the weaknesses of this strategy the first time that an unsupervised 5-year-old with a crayon converted drawings of antelopes into helmeted space travelers, thereby confusing a generation of New Age astronomers in 1960s California and producing the Chariots of the Gods mythology that lives to this day.

  • Some centuries later, the Greeks used stone tablets to record their thoughts, previously relegated to unreliable oral history. It made sense to one early philosopher who observed, "Heavy thoughts require a heavy medium." But problems with data portability and physical storage precluded the Greeks from writing anything terribly lengthy. This probably explains why neither daily newspapers nor dime novels were Greek innovations.

  • The Egyptians were among the first civilizations to dabble in paper, but the vulnerabilities of this medium were evidenced quite clearly when Julius Caesar set fire to the Alexandrian Library in 47 BC. Fire also proved to be the foil for wax tablet-based data storage in the later stages of the Roman Empire. In fact, the world got its first experience with a data storage meltdown as Nero fiddled in the mid-60s AD.

    In the Middle Ages, attention turned from the medium to the method. The rise of monastic "data centers" -- and the approach to protecting and preserving a centralized information store using a disciplined cadre of well-trained monks -- helped define most of the standards for modern data storage management. IBM traces its origins here, and resurrected the monastic model in corporate client shops throughout much of the 20th Century. Arthur Anderson might also have borrowed from this model, seizing upon those practices for concealing privileged information from the unwashed and shredding documents containing embarrassing or heretical information.

    (Interestingly, many contemporary concepts for securing data also originated from this period, including the use of a moat -- the equivalent of a modern firewall -- for protecting information from outsiders, and secret codes -- a.k.a. key encryption -- for exchanging information securely between monarchs. But that is another history...)

    With the rise of distributed data, beginning with the creation of the Gutenberg press, the information genie was out of the bottle. Data ran amok outside of the carefully controlled confines of the monastic library, challenging the ability of either the Dewey Decimal System or the Hypertext Transfer Protocol (HTTP) to keep up.

    In the absence of a single dominant "censor" (a role for the Roman Catholic Church in Medieval Times), information proliferated using a plethora of languages and semantics. Truth be told, the anarchists, nihilists, and existentialists -- and not former Vice President Al Gore -- created the Internet and World Wide Web. In the process, storage management became exactly like the state of nature as conceived by the modern philosopher Thomas Hobbes, a place where life was nasty, brutish and short.

    The central issues of contemporary electronic storage management haven't changed a lot since the Modern Era. The rise (and fall) of kammeralist empires, with their zero-sum-game weltanschauung, and of sea-faring colonial empires with their mercantilist mentality, have contributed little to the problem but a price tag. Today, as in the past, the problems are the same:

    1. The One and the Many: Storage management is clearly simplified by the use of single source products. Homogeneous storage platforms, servers, file systems and proprietary management solutions combine to create a more highly manageable storage environment. However, such solutions obviate access to best-of-breed technologies that may develop outside the single vendor cloister -- technologies that may deliver a cost-savings, risk reduction or business enablement advantage to competitors who use them. In short, vendor lock-in can be tantamount to slow death.

    2. Nature versus Control: In keeping with the trends of modern history, most storage management vendors (and consumers) have concerned themselves less with understanding the nature of data storage itself, than with the engineering problem of how to control storage. As a result, more attention has been paid to tactical (and practical) fixes to short-term issues than to strategic approaches than cut to the heart of management issues over the long haul. We have been less interested in divining ways to truly share a single copy of data, and instead focused our full attention to finding faster ways to copy data for the purposes of sharing -- even at IBM's Data Sharing Competency Center. The end result of this preference for storage engineering over storage understanding is the quagmire of unmanaged storage growth and fast equipment obsolescence that confronts IT managers today.

    3. Reality versus Perception: Storage management has been reduced to a perception issue. Competing approaches are discriminated from one another based on the thinnest of differentiators, which is why, to paraphrase Henry Kissinger, the battles between vendors are so bloody. The latest manifestation of this phenomenon has been the introduction of Fibre Channel fabrics, incorrectly labeled Storage Area Networks. Fibre Channel is not a network protocol, so it cannot be used to produce a network. Most importantly, FC lacks any provision for in-band management, so the fabrics created with FC do nothing to advance the goal of storage management. Yet, every vendor offering a "Fibre Channel SAN" insists in its brochures that SANs solve storage management issues associated with server-attached storage. It is just another example of a lie that, when told often enough, achieves the status of 'truth' -- from a marketing perspective, at least.

    On the final point, another observation should be kept clearly in mind. Before a networked storage strategy can contribute anything to storage management, much work will need to be done in terms of software enablement. Today, despite the claims of SAN vendors, storage networking technology is incapable of providing the ideal of a Baskin-Robbins ice cream shop, offering many different flavors of data storage. They offer only block storage of the most rudimentary type, and with zoning and virtualization, an extension of the paradigm of a partitioned server-attached array. IT managers are wise to continue to rely on the time-honored policy of discerning, based on the application that is being deployed, what type of (and how much) storage is required.

    Only then should they consider the various storage platforms and topologies that can meet the requirements and select those platforms that can be effectively instrumented for monitoring via an application-centric approach (BMC Software's Patrol comes to mind as a smart framework-based approach). Storage networks are not a panacea and will not be until several layers of software are stacked atop each other that enable the automatic delivery of just the right kind of storage at the right cost to the type of data that must be stored. That kind of intelligence is years away.

    I hope this much abridged, and somewhat gonzo, history of storage management is helpful. At a minimum, it should make for good cocktail party foo at the next SearchStorage conference reflecting, as it does, the benefits of a solid liberal arts education from any of America's best-run universities.

    About the author: Jon William Toigo has authored hundreds of articles on storage and technology and authors the monthly searchStorage "Toigo's Take on Storage" expert column. He is also a frequent site contributor on the subjects of storage management, disaster recovery and enterprise storage. Toigo has authored a number of storage books, including, The Holy Grail of Data Storage Management.

  • Dig Deeper on Data center storage