Best Practices: Tackling data migration

Data center projects often involve migrating data, which is frequently a painful process that can lead to unplanned downtime and outages. It's time to adopt consistent, repeatable migration practices. Selecting the right approach is highly dependent on infrastructure limitations, data and platform types, time constraints and staff capabilities.

This Content Component encountered an error
This article can also be found in the Premium Editorial Download: Storage magazine: RAID turns 20: Do you still need it?:

Successful data migration projects require as much planning as doing. Use this checklist to make sure you have all the bases covered.


Data centers are active places with no shortage of large-scale projects: the merging of multiple sites, green IT initiatives, virtualization projects, server consolidation and the perpetual cycle of technology refreshes. A common thread throughout all of these endeavors is the migration of data. From a storage management perspective, data migration has traditionally been treated as an exception to normal operations. Data migration also seems to coincide with unforeseen difficulties that lead to extended downtime and the need to scrub, roll back and reschedule the activity.

In larger environments, data migration is no longer an occasional disruption but a regular activity that consumes an increasing number of staff hours. As with seemingly everything else related to infrastructure, there are many technology options from which to choose. Selecting the right approach is highly dependent on infrastructure limitations, data and platforms types, time constraints and staff capabilities.

Given this rise in migration activities, as well as the critical and often highly visible business impact of failed migrations, it's time to adopt consistent, repeatable migration practices to support various data center initiatives. But various requirements within a dynamic multifaceted environment may sometimes conflict with one another. Given this reality, how does one begin to develop a standard migration methodology?

Practice makes (nearly) perfect
The migration methodology should address the planning, pre-migration, migration and post-migration phases and include (or reference) detailed information within each phase, such as flowcharts and procedural documents. The degree of specificity within these documents should be high and include platform-/OS-/network-/SAN- and storage- specific tasks. Detailed guidelines should be included for verification and validation at each level.

Planning. The most important consideration for any migration is the overall business impact and, specifically, the impact on an application and its ability to access data. For this reason, the planning phase is the most critical. In a large migration project, the planning phase can and should consume more staff hours than the actual migration itself.

It's important to understand where data migration fits within the larger project. Is this a data center relocation where data may have to be migrated over long distances? Part of a consolidation effort focused on maximizing storage layout for efficiency while still ensuring appropriate service levels? Or is this a lease expiration or technology refresh involving migration of data from one generation of storage hardware to the next? Data migration can also be part of a performance optimization or tiered data effort.

The heterogeneity of the environment, in terms of servers and storage, has a significant impact on the migration process, particularly in determining the numbers and types of tools available to perform the movement. Likewise, the scale of the migration in terms of numbers of hosts impacted and the volume of data to be relocated are critical factors affecting how the migration is accomplished.

Minimizing business and application impact usually translates into ensuring the smallest outage and the least performance degradation, so a key element of the planning effort is determining how to minimize migration time. This often involves organizing interdependent apps into move groups that will be migrated together.

Another key part of planning is the verification and validation process. What steps will be taken to ensure that each phase of the migration has succeeded? It's equally important to plan (and test) the fallback process.

Pre-migration. Migration prep consists of detailed activities to ready the server, network and storage systems impacted by a given migration. Activities include:
  • Backing up data
  • Capturing current configuration information
  • Detailed review of all planned changes at each level
  • Identification of migration validation checkpoints
  • Script configuration and review (if used)
  • Review/verification of LUN/volume source and target information
  • Review/verification of pre- and post-migration OS-level drive mappings
  • Review/verification of post-migration app reconfiguration
  • Review/verification of clustering modifications
  • SAN reconfiguration and rezoning
  • Storage preparation
  • Host preparation (dependent on migration approach)
Migration. Data migration can be accomplished at several levels. The selection depends on the number of servers and storage devices, the quantity of data, the distance involved and the tolerable level of impact to apps.

Host-based: The traditional means of migration, pre-dating storage networking, is host-based. Approaches include app-specific migration, database replication, the use of OS-based or third-party volume managers to mirror and split volumes, general OS file-copy utilities and special-purpose migration apps. Each has its particular advantages and disadvantages. The overall benefit of host-based replication is that it can support heterogeneous storage and be economical for smaller scale requirements. General disadvantages are the intrusiveness and impact on the host, and overall scalability when lots of systems are migrated.

Storage-based: Migrating at the array reduces host impact, but has traditionally been limited to homogeneous transfers, such as moving between generations of a vendor's product line--a reasonable option if you happen to fit this scenario, but a non-starter if you don't.

More recently, the situation has changed. Specifically, Hitachi Data Systems' (HDS) TagmaStore controller-based virtualization product can support a variety of non-HDS arrays making it an excellent option for migrating data. EMC offers its Open Replicator for Symmetrix, a resident utility that can pull or push data to non-Symmetrix systems on a SAN thereby enabling "outside-the-box" data movement.

Network-based: What if you need heterogeneous replication and don't have a TagmaStore or a Symm? Many tools, both appliance and switch-based, as well as SAN- and NAS-based, have emerged. On the SAN front, Cisco Systems and Incipient offer switch-based migration and replication tools, provided you have an application processor in your Fibre Channel director. Among the SAN-based appliances available are the Brocade Data Migration Manager, FalconStor Software IPStor and IBM System Storage SAN Volume Controller. In the NAS arena, EMC Rainfinity and F5 Acopia ARX Series appliances can be deployed to provide transparent file-based migration. These options minimize host impact but provide flexible movement across various platforms.

When it comes to developing a consistent, generally applicable methodology, the most common considerations are heterogeneity, the ability to deploy and remove with minimal effort, local and remote support, and ease of management.

Generally, the best options would be a host-based app that runs on a wide range of platforms or, to minimize host disruption, a SAN- or NAS-based virtualization appliance.

Post-migration. These activities focus on data verification and validation at the storage, OS and app levels, and should include low-level data integrity checks and higher level app accessibility and performance verification. A review of relevant procedural documents, as well as asset management and configuration tracking repositories, should be included to reflect the changed environment. A migration close-out should also take place to review the process, resolve issues and provide feedback for improving future migrations.

How big can the migration problem become? We've seen sites with brand-new, enterprise-class storage arrays sitting idle for many months due to migration challenges. Extending leases and having assets sitting idle aren't typically factored into TCO calculations. Avoiding these situations with a standard migration strategy can be a solid investment.

This was first published in November 2007

Dig deeper on Data management tools

Pro+

Features

Enjoy the benefits of Pro+ membership, learn more and join.

0 comments

Oldest 

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to:

-ADS BY GOOGLE

SearchSolidStateStorage

SearchVirtualStorage

SearchCloudStorage

SearchDisasterRecovery

SearchDataBackup

Close