Staging older files to the SATA partition

Staging older files to the SATA partition

I am using Solaris 9 on Sun servers with Hitachi storage system (on SAN with Brocade FC switch). I would like to add a few SATA terabytes to the storage system. I'm looking to do "disk staging" so that all the files from my partition older than two weeks or bigger than 10 MB will be staged to the SATA partition. These operations must be transparent to the application. I'm using VxFS and VxVM now.

    Requires Free Membership to View

    Login

    When you register for SearchStorage.com, you’ll also receive targeted emails from my team of award-winning editorial writers. Our goal is to keep you informed on the hottest topics, the latest news and the biggest challenges you face as a storage professional today.

    Rich Castagna, Editorial Director

    By submitting your registration information to SearchStorage.com you agree to receive email communications from TechTarget and TechTarget partners. We encourage you to read our Privacy Policy which contains important disclosures about how we collect and use your registration and other information. If you reside outside of the United States, by submitting this registration information you consent to having your personal data transferred to and processed in the United States. Your use of SearchStorage.com is governed by our Terms of Use. You may contact us at webmaster@TechTarget.com.

Surprisingly, this is not easy to accomplish. In order to have some files on one storage system and others on another, you either have to use two different filesystems (and volumes and volume groups) and move the files with a script or SRM package, or you have to use an HSM (Hierarchical Storage Management) product.

One free method I've seen is to use a script to move the old files to your nearline/ATA storage and leave Unix symbolic links behind. Let's call this manual HSM. This works surprisingly well in some circumstances, especially if there are relatively few files to be managed in this way. But watch out -- each symbolic link requires an inode in the filesystem, and you can quickly run out of inodes! df_vxfs will show you how many inodes are left, and if you will have more than 8 million files, you need to use the "largefiles" option of mkfs when creating the filesystem. And look out for applications that aren't friendly to symbolic links, or none of this will work!

This was first published in August 2004