Automate data migration


This article can also be found in the Premium Editorial Download "Storage magazine: Salary survey reveals storage skills are in demand."

Download it now to read this article plus other related content.

HSM Software

Requires Free Membership to View

Click here for a listing of HSM software products (PDF).

Stub files and inodes
Once the HSM software is configured to work with the different types of media in your storage environment, HSM software products use either stub files or inodes (in a Unix-based OS, an inode is a stored description of an individual file) to track the location of migrated files. Most HSM software products create stub files that are approximately 4KB in size. These files may contain information such as the new physical location of the file, file attributes such as last-accessed date or security permissions, and the file's header information. Some products, like SGI's InfiniteStorage DMF, use inodes created by the host's file system. For DMF, inodes are 256- or 512-byte data structures created for each file by SGI's CXFS file system. DMF then accesses a field reserved for it within each file's inode and sets a flag to indicate if the file has been migrated.

While stub files and inodes have similar purposes, each HSM product varies in how it creates and manipulates them. For instance, when CA's BrightStor HSM migrates a file, rather than using the native file-system copy or move utility, it uses its own copy utility that's installed with its agent software. The utility identifies a file to be migrated, locates its intended destination, copies the file to that location and then creates a stub file on the primary disk that includes the file's new physical location. Before deleting the original file that's still on primary disk, it verifies that the file copy is good.

SGI's InfiniteStorage DMF uses a similar process to migrate the file, but unlike CA's BrightStor HSM, it stores the file's location in DMF's central meta data server and changes the flag setting in the file's inode. When a request for the file is made, the CXFS file system recognizes from the file's inode that the file is no longer on local disk, so it makes a call to the DMF meta data database server for the migrated file's location.

As part of the stub file creation, HSM products like those from CA and EMC let users set the size of the stub file and its contents. The rationale behind this is to control how much of the original file's header information an administrator wants to retain on primary disk. For instance, an administrator who needs to manage a large number of files containing photos or videos may include descriptive information about the photo or video as part of their file content. Rather than migrating the entire file and keeping just the file attributes, the stub file will retain this information, which can be searched along with the file attributes. This expedites searches for specific files, frees space taken up by infrequently accessed file content and prevents massive recalls of file data when searching for just small portions of the file.

Users of EMC's DiskXtender can take advantage of another product option--its high watermark threshold. This feature migrates the file and creates the stub file, but keeps the original file on primary disk until disk capacity reaches a certain threshold or watermark. This lets users keep their data on faster, primary disk until this threshold is reached to facilitate faster file recall. Once the threshold is reached, DiskXtender kicks in and starts to move files, but only after updating each stub file to point to the migrated file's new location. This process of updating stub files and deleting files on primary storage continues until DiskXtender reaches a low watermark or the last file to be migrated.

This was first published in November 2006

There are Comments. Add yours.

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: