Published: 07 Jul 2005
The process of changing storage platforms is known as a forklift upgrade because everything associated with the old storage platform has to be physically moved to the new platform. Data, connections and authorizations must travel to the new environment; servers, applications and databases also need to be redirected to the new storage devices. The procedure takes a great deal of time and Murphy's Law usually rears its ugly head: Whatever could possibly go wrong more often than not does.
The prospect of a forklift upgrade--switching from one vendor to another--discourages companies from changing storage platforms. Many storage administrators would rather push their existing storage to the max, even past the point where it becomes obvious that a change is needed, just to avoid changing vendors.
Sometimes, there's no choice. "We had gotten to the point where we simply outgrew our small EMC storage area network. We couldn't expand it any further," says Mark Rivard, network systems specialist at Johnson Health Network, which is located at Johnson Memorial Hospital, Stafford Springs, CT. A pressing need to expand its storage, combined with frustration over the cost and complexity of maintaining a Fibre Channel (FC) SAN, drove Rivard's organization to undertake a forklift upgrade to an iSCSI SAN from EqualLogic Corp.
Since vendors began focusing on scalability upgrades, much of the pain associated with an upgrade to a bigger box within the same vendor's product line has been minimized. The reason is that when switching to a larger or smaller model within the product line, the system often automatically moves everything intact. But changing storage vendors remains a challenge, with forklift upgrades still the norm.
Despite the challenges involved, organizations are forced to change storage platforms all the time. Storage spoke with several storage managers who've recently gone through a major storage platform change and had them identify critical "gotchas" and best practices for moving from one vendor's offering to another's.
Moving data to the new platform is just one of the challenges involved in changing platforms. "You can always use backup software to back up the data and restore it on the new array, but it's kind of a brute force solution," says Steve Jeffreys, engagement partner at GlassHouse Technologies Inc., Framingham, MA.
Jeffreys recently moved a client's storage from an IBM Corp. Shark to an EMC Corp. DMX. He used a combination of backup and restore, reconfigured Shark host bus adapters (HBAs) for the DMX and mirroring. Other challenges users will confront when changing platforms include preserving the meta data associated with the data, he says. There are also the tasks of mapping volume sizes from one platform to another, redirecting apps and databases to the new storage, configuring mirrors, RAID levels and other technical details.
Security is often overlooked, and proved to be a key issue for Johnson Health Network. "We have thousands of network shares with varying levels of security," says Rivard. The hospital turned to EMC's FullTime RepliStor (formerly Legato RepliStor) to copy all of the permissions to the new storage environment while maintaining synchronization with the old one. "It allowed me to change the location of various folders and data while maintaining security," he notes.
EMC Symmetrix to Hitachi Data Systems Thunder
The move from EMC to Hitachi Data Systems (HDS) Inc.'s storage could have been much more difficult for Drummond Co. Inc., a Birmingham, AL-based energy and coal-mining company, but it used EMC with only one application--a big, critical PeopleSoft system running on a Sybase database. To ease the transition, Drummond scheduled the change to coincide with the end of its Symmetrix lease. This gave it 60 days to experiment with the new platform before putting it into production.
"We started with a free, 30-day, onsite trial," says Kirk Pierce, Drummond's technical support manager. During the trial, the team pushed the Thunder array to find its performance limitations. It passed the performance tests and gave Pierce's team the experience it needed. Drummond then negotiated to buy the system, which gave them another 30 days to work with the storage while the negotiations and paperwork proceeded.
The key to Drummond's migration was a three-step process to prepare the new storage for production that began the moment the deal was finalized. "The first step was defining and allocating the RAID groups," says Pierce. Earlier testing had revealed that the firm wanted RAID 5 for the PeopleSoft database, using six drives for data with another drive storing parity information. HDS engineers worked onsite and set up the RAID configuration using HDS tools.
"The second step was building the actual LUNs [logical unit numbers]," Pierce says. HDS engineers worked with his team to show them how to do this. Since then, Pierce's team has added LUNs on its own.
The third step was creating the FC SAN using two Brocade switches. Previously, the company's EMC storage had been direct-attached. Again, HDS engineers helped Pierce's team to set up the initial zones; they have since added zones without outside assistance.
With the preliminaries out of the way, Drummond planned to shut down transaction activity for a weekend and move to the new storage platform. To move the bulk of the data, the team let the Sybase database do the heavy lifting. "Our DBA mapped it all out to make sure the right data would go to the right place," says Pierce. Then, using Sybase's own database dump and restore utilities, the DBA sent the PeopleSoft-Sybase data to the HDS platform using the FTP protocol.
To set up multipathing through its two Brocade switches, Drummond used Hitachi's Dynamic Link Manager (HDLM), which provides high-availability failover and limited load balancing. Using HDLM, the storage team defined the primary storage paths for each switch. From there, HDLM took over.
|Five ways to move data across storage platforms|
Network Appliance to EMC Symmetrix
Fast-growing Overstock.com, an online merchandise discounter in Salt Lake City, deploys multiple clusters of Dell servers running Linux and an Oracle database. Since September 2004, it has been moving clusters of servers as quickly as it can from a Network Appliance (NetApp) platform to its new 70TB Symmetrix. Overstock.com felt forced to undertake the forklift upgrade for two reasons: The NetApp filers couldn't handle the workload, and it was unhappy with vendor support.
For Overstock.com, the trick was moving to the new storage while maintaining 24x7 operations. The Web site is taken down only once a year, says Shawn Schwegman, Overstock.com's VP of technology, and the biggest challenge was moving the Oracle database.
"We installed Oracle on a new cluster of servers connected to the EMC. We then exported the data from the old cluster to the new cluster using Oracle's hot standby capability. Then, we restored the cluster on the new database," says Schwegman. The migration team then spent the next few hours copying transaction logs to the new database cluster. "From the database perspective, this is the easiest way--the way Oracle recommends," he says.
An alternative approach is to shut down the database entirely, back it up all at once and then restore it on the new platform. "If you have a smaller database, this works well," says Schwegman. Overstock.com's problem was its multiterabyte database, which would require more than a day to copy the logs over the network. Overstock.com tried the recommended standby database approach in a test environment and found that it took only four to six hours to bring up the standby database.
Having resolved how to move the critical database cluster by cluster, Overstock.com was then comfortable letting EMC direct the rest of the migration and configuration. EMC analyzed the storage on the NetApp box, which revealed hot spots among the LUNs. Based on this, it recommended distributing the database differently.
"We spread those LUNs over more disks because certain tables were getting hit too much," says Schwegman. Overstock.com also accepted the EMC read-ahead cache algorithm, which delivered better than a 99% cache hit ratio, higher than it achieved with NetApp.
EMC also helped set up the zones and masking. "Usually, we'd never allow a vendor to do this, but our people were looking over [EMC's] shoulders all the time. Now our people are trained and we can do this [process] ourselves," Schwegman says.
As clusters moved from the NetApp platform to new clusters connected to the EMC platform, Overstock.com wiped clean the servers previously connected to NetApp, swapped out the network cards for SAN cards and put the servers back in service. "We've got it pretty well scripted and documented," says Schwegman. The company can now turn over a six-node cluster in a day.
EMC to BlueArc NAS
After repeated attempts by EMC to fix its aging storage array proved unsuccessful, Mark Chandler, director of information systems at Cambridge, MA-based NeoGenesis Pharmaceuticals Inc., decided to switch storage platforms. The first crisis was what to do with the company's incessant data stream coming from its lab instruments while its primary storage was down. "We were juggling for a week, putting the data on different servers, wherever we had space," he says. At one point, NeoGenesis stopped collecting new data and worked only with what it had.
The second crisis was moving the data from the EMC box to its initial 2TB of new BlueArc Corp. NAS storage. "The EMC [array] had locked itself up trying to recover, so we couldn't get at that data," Chandler explains. Instead, the company reverted to older tape backups, recalled from an offsite archive facility, and restored the data directly onto the new NAS box. NeoGenesis used a 25-tape LTO backup system.
Configuring the new NAS device wasn't a problem due to upfront preparation. Chandler's team spent considerable time planning the details of how the raw storage space was to be configured. The configuration they decided on was then built at the factory. "When it arrived, we put FC on the back and the control head saw the storage right away," he says.
Redirecting NeoGenesis' devices and servers to the new storage was straightforward. "BlueArc looks like a standard network file system share. It registers itself in Active Directory like any other server," says Chandler. And because the company runs CIFS and Unix shares, all NeoGenesis needed to do was modify two lines of code and everything went where it was supposed to go.
Changing storage platforms is getting easier. With planning and help from the new vendor and consultants, a platform change can be manageable. Although it's not yet as intuitive as loading songs onto an iPod, switching vendors isn't the hassle it once was.