Taking stock of data archiving and retrieval practices
We're trying to understand the best practices for data archiving and data retrieval.
1. We have to have an archive infrastructure that is hierarchical (business lines up to parent co. up to corp.)
2. We have to keep all transactional sales data for royalty reporting and licensee reporting.
3. We have to keep lots of financial data.
4. We have a diverse DB infrastructure (some with AS/400, Oracle, SAP, SQL, and such).
Any ideas where to get started?
I believe the most important thing to do is to take stock of the processes and transactions that you have in place and then look at a specific process for re-engineering.
What happens in most cases is that many IT organizations get ahead of themselves and try to re-engineer the large processes without taking a smaller approach and focusing on a best practice smaller process as a test. The goal of trying a smaller project out is to mitigate risk in a new process and find out what process re-engineering the group can handle before trying to tackle larger, more noticeable. You will need to look at the goals upfront and understand what the improvements you want to make are and put that into a quantifiable measurement.
For example, the company requires a restore in 24 hours. How often does the IT organization make this requirement, 50% of the time, 60% of the time? Then when you migrate to a new methodology does the new methodology increate the mean time to recovery or decrease it? You now have a stated goal up front. Look at the stated goals in terms of cost, complexity, performance and time. Each of these variables can determine the effectiveness of the overall solution.
Also, involve your end-customers in the re-engineering process since they may see their needs differently then the IT organization and at the end of the process the end-customer is the one who will judge if the solution is effective and flexible enough to meet their needs.
One thing to look at is adapting the tape arrays with a front-side disk cache. In this way one can consolidate the current tape environment into an archiving infrastructure and focus the disk-based backup
technologies on the day-to-day backup operations, saving archival for a weekly or monthly activity (all depending on your requirements; don't forget about the federal government requirements
for off sites). The nice thing about disk-based backup is that it adapts to the current environment without causing any major unplanned downtime as many of the leading disk-based backup applications treat disk-based backup devices as any other logical storage device. You may want to work with your storage provider of choice to understand the specific trade-offs in your environment.
Editor's note: Do you agree with this expert's response? If you have more to share, post it in one of our
This was first published in January 2004