The inevitable wincing is immediately noticeable whenever the subject of data migration comes up. Most IT professionals have vividly painful stories surrounding data migration gone awry. The exercise of a data migration project can crush many IT operations and decimate the relationship with the storage vendor. Done correctly, it can cement a long, mutually beneficial association.
Data migration is a requirement whenever a storage system is replaced and when storage systems are consolidated. Other times it may be required when a storage system is added or upgraded.
So why are there so many war stories about data migration gone wrong? Is it technology? Sometimes the technology is the culprit. More often it is the human missteps, not the technical ones, that cause the headaches. Missteps could include: poor project planning; lack of criteria for what makes a successful data migration; misrepresented user expectation setting; data sets that are poorly classified or only partially classified; compatibility issues and vendor knowledge gaps (most vendors lack knowledge and experience with other vendors equipment); incomplete security and data protection requirements established; application disruption is greater than anticipated; the budget is under-funded; and timelines are severely underestimated.
Data migration is a complex and important storage project. It should be treated as a high priority and managed with discipline. As with most storage technologies, data migration products come in three flavors: host-based, network-based and storage-based. Regardless of the chosen flavor, successful, "painless" data migration requires best practices that establish:
- The window of time committed for the data migration to be completed;
- The classification of the data sets. Data classification also articulates data set accessibility, security (such as encryption) in flight and/or at rest, retention and destruction requirements;
- Which data sets are to be migrated and to what volumes;
- The service level agreements and the level of application disruption allowable. Can it be completed non-disruptively online, or must it be accomplished disruptively offline because of time constraints?
- The methodology that will be acceptable;
- The responsibilities, roles and tasks for each individual and department. If professional services are contracted, expectations must be clearly and completely expressed (and agreed to in writing);
- The definition of a successful data migration. The right data is migrated to the correct target volumes within the timeframes committed with the resources allocated that matches all service level agreements.
There are many products in the market today (with more being announced all the time) specifically aimed at reducing data migration pain. Many of the newer SAN and NAS-based products are designed to make data migration a non-event. The technologies leverage virtualization at either the block or file level, designed to make data migration a totally application non-disruptive process.
This is no doubt a great boon for data migration; however, caveat emptor [let the buyer beware]. Failure to follow data migration best practices can still lead to the wrong kind of memorable experience.
About the author: Marc Staimer is president and CDS of Dragon Slayer Consulting in Beaverton, Oregon.