If data is not backed up, it cannot be recovered. The concept is simple. However, determining whether a backup has succeeded is the number one challenge facing managers of backup operations, according to a recent InfoStor poll. Despite investing millions of dollars on backup systems to ensure the availability of mission critical data, the typical organization still can only take an educated guess as to whether and what extent backup operations are actually protecting data, let alone meeting specific reliability and efficiency objectives.
This is where the concept of gathering "storage intelligence" about your backup operations becomes particularly useful.
What is storage intelligence?
Products that perform storage intelligence -- whether provided by a third party or developed in-house -- actively collect, consolidate, analyze and present critical information about the success of your company's data storage operations. By obtaining adequate storage intelligence, you can more reliably and efficiently determine how well your backup procedures and data storage operations are complying with your company's enterprise policies. At the core of storage intelligence is the collection of data generated by systems about their operations.
Applied to backup environments and analyzed in the proper context, storage intelligence data from backup systems can help:
- Identify errors (e.g., backup failures)
- Identify performance issues (e.g., bottlenecks)
- Identify the root cause for each reliability and performance issue
- Provide an auditable track record of all backup activities
- Identify sources of demand for backup services
- Identify and predict resources needed to provide backup services
- Enable organizations to accurately bill for the cost of backup services
It also can help with implementation of new backup systems, permit more efficient consolidation and migration of backup systems, train administrators and much more.
Managing the complexity
The typical enterprise backs up numerous applications, blocks and files residing in a mixture of Windows, Unix and Linux network operating systems. The typical enterprise also backs up these items across heterogeneous networks to multiple tape devices using backup management software from multiple vendors. To make matters worse, the rapid growth in data volumes, the centralization of management of backup operations within organizations, and the acquisition of new businesses have forced many organizations to scramble to integrate incremental backup products and systems to an already intricate environment.
The result is a complex, volatile network of heterogeneous backup and storage technologies that can quickly degrade without proactive day-to-day management. The resulting complexity is one of the key reasons why the radical decrease in the per-gigabyte cost of hardware has not led to a corresponding decrease in the cost of managing these systems. In fact, for many organizations, administrative costs have continued to skyrocket while the reliability and performance of backup systems has decreased.
Implementing a storage intelligence solution in your backup environment
Three basic steps are required for implementing a storage intelligence solution:
- Define the problem to be solved (for example, identify all failed backups, etc.);
- Identify and collect the required data from one or more aggregation points a backup server, a tape library, a network node, etc.); and,
- Analyze and present the collected data in a form or format that helps to pinpoint and resolve the problem (that is, turns the data into information or even knowledge).
Unfortunately, the information vital to measuring the reliability and performance of backup operations is often buried within vast log files or databases. It's also often mixed with massive amounts of non-critical data. Manually sifting through, collecting and consolidating these records is a tedious and time-consuming task that can take hours for even a small backup operation that manages only a few dozen protected resources.
In larger environments with multiple servers, operating systems and storage architectures, manual collection and consolidation of key operational data can be impractical. As a result, larger and more complex environments require an automated process to collect, consolidate, analyze and present key storage intelligence information.
Whether the storage intelligence solution is provided by a third party or is internally developed, it should meet the following minimum criteria:
- Implementing the solution does not increase the risk, complexity or volatility of the existing backup environment.
- Deploying and upgrading the solution is quick and easy (think minutes or hours, not days or weeks).
- Implementing the solution does not require the deployment of additional software or technology on any backup servers, nodes, libraries or protected resources (such as agents).
- All data is presented in a way that intuitively enables administrators to identify and solve identified problems quickly and easily.
- All reporting functions are easily customizable to reflect the way the enterprise manages its backup environment (for instance, it might allow customization to be backup server-centric, tape library-centric, centered around the owner of the resources being backed up, etc.).
- The solution consolidates intelligence from the organization's existing heterogeneous backup environment, including multiple backup products, networks and devices across multiple operating systems.
- Additional backup servers, targets, clients and tape libraries are automatically or easily added/deleted as the environment changes.
- All data is stored persistently in an open, industry-standard database (such as, Microsoft SQL Server) and is accessible with standard database query and reporting tools.
- The solution scales with continued growth in corporate data, protected resources and applications.
- The solution is supportable in the field by the vendor.
A storage intelligence solution is required if an organization desires to establish, measure and proactively manage backup operations against enterprise storage policies. Furthermore, it is impractical, if not impossible, to effectively manage a heterogeneous backup environment without such a solution. Even a basic storage intelligence solution will substantially reduce storage administrative costs and at the same time reduce the risk of lost and unrecoverable data.
To learn more about storage intelligence, consult your backup expert. You can also view Bocada's storage intelligence product from our Web site.
This was first published in August 2002