Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Storage management tools/policies -- Which comes first?

Unix Systems Administrator John Vignoni recommends ways to establish storage management policies in an IT organization.

Commentary: Unix Systems Administrator John Vignoni had some interesting insights to share with SearchStorage recently on how he is approaching the establishment of storage management policies in his IT organization. Readers might be surprised to learn that he sees the storage management tools as a very important, but secondary, component to this process.

More importantly, Vignoni remarks that the process must begin with end users of a company's data. The usage of SRM tools and features must always be applied first through the "filter" of a company's users and how they most like to operate. Do you agree or disagree with his assertions? How does what he describes relate to your own organization? Read his comments, then rate this tip, and .ekaEatXggsM^10@.ee83ce3/309!viewtype=>add your own input.

How do you empower users to start managing their own data? How do you get them started?

At Lockheed Martin's Palmdale, Calif.-based Enterprise Information Systems group, we chose the equivalent of giving the end user a pretty big "file cabinet" to archive their data: In our case, it was via a centralized storage management tool, Tivoli Storage Manager (TSM), which we use for both archiving and backups. We also included a custom, client-side archiving tool that we place on the users' desktops to help them perform their own archiving, track when archiving will occur and what they've already archived.

Working with end users on data management issues

So, how do you convince end users to manage and begin archiving their own data? How do such features as archiving, auto-archiving and quotas work with end users? Here are some analogies and scenarios which may help.

Data management can be equated with managing your home. Case in point, you have a drawer in the home where you keep your current and/or yearly bills. When the bill is paid and/or the year ends you may move those items to another place - a file cabinet (archive), the trash (delete).

If the file cabinet gets full, you will generally clean out (delete) the old stuff that you don't need and perhaps keep some stuff (keep in archive) that you need to hold on to. You would not say to yourself "I can't take the time to go back and look at all my old bills/paperwork." You'd find the time to clean out the file cabinet to make room for new items. You would not go out and buy another file cabinet. The same principles must hold true for managing data. In our case, the benefit is that the end user is given a pretty big file cabinet.

Working with quotas. Living with storage management quotas is also a lot like living in a house. You can only afford a house so big (500M Bytes). You put things in it (files) and take care of it by managing what's in it and how it looks. The same is true for your data -- the company you work for can only afford so much disk space. You take time to put things in files and directories. You also need to take time to manage what's in it and how it looks.

One thing about quotas, though, is that end users must be given the tools and procedures on how to manage their data first, before the quota is implemented. Don't implement quotas, and then tell the users to delete files or find a tape drive or someone with a CD burner on which to save the data. Frankly, this is not fair to the end user.

Automated archiving. On first glance, automated archiving for inactive files also sounds like a good alternative that could save end users some time. However, when we presented this to the client community, immediate questions and concerns were raised about, "How will I know my data has been moved?", and "I may need that some day, so don't move it."

Automated archiving should obviously be driven by end-user requirements. In the absence of people coming forth with their own data retention requirements, it's tempting for IT to implement a policy that states, "Files not accessed in X number of days will be archived for X number of days." However, this is difficult because it is the users' data and they must determine how to manage it.

Communication is key. The biggest challenge in establishing successful data storage management procedures remains communication. The biggest hurdles comes in making people aware of what the tools are, how they are used and what the tools do to their data.


We then ask end users to take time to organize their data, as they would when organizing a file cabinet in their own home. After having analyzed our existing data problems, concerns and needs, we identified this aspect -- getting users to manage their own data and communicating the need to them -- as the most crucial components in the storage management puzzle. Implementing quotas should also help to drive end users to use the archiving tools we've set up to manage their data.

In our process of developing an appropriate storage management strategy, we've also explored related issues like setting quotas, auto-archiving and hierarchical storage management (HSM) to see how they would fit within the end user culture, and to learn how they'd best meet the archiving needs of our users and organization. The sidebar story shares some thoughts on ways to communicate to end users.

How we implemented our data archiving/backup solution and evaluated the use of other data management features like HSM

As previously mentioned, we use Tivoli Storage Manager (TSM) for both centralized archiving and backups.

We also developed a front-end archiving tool placed on the client's desktop that allows archiving to take place at the client's request.

This custom, client-side tool now provides users with many answers to their questions right from their desktop, such as:

  • Which type of data will be archived
  • For how long it will be archived
  • When the archive will run (at a specified interval, such as daily, weekly, etc.)
  • Which specific files have been already archived by the user (via a log file)

Prior to this implementation, many users took advantage of other tools for archiving, such as zip drives, CD burners and even locally attached tape drives. It's one thing to have a user work with another piece of software, let alone handle the media and manage the storage of that media.

Our current, custom desktop tool now eliminates the need for the user to work with and manage separate media, and reduces hardware costs because there is no CD burner or zip drive being purchased. Other benefits have included the media being handled and managed by a centralized organization (computer operations) as well as the elimination of media being lost (a common problem experience when users were writing to CDs and zip disks). It also gives the end user community a virtually unlimited amount of storage space as there is always an ample supply of tapes.

End users now also use the archive feature in TSM for archives of larger files as follows: They contact the help desk and open a trouble call identifying the data that needs to be archived. The call is then assigned to computer operations at this site, and the established procedure handles the archive. Even with the implementation of an archive tool on the desktop, this process is still considered important for archives involving large amounts of data, perhaps 10 to 20G Bytes or more.

There are several benefits to using TSM for this. Computer operations is already trained and familiar with the movement of tapes through TSM. The implementation of the archiving feature is transparent to them. Those who are familiar with TSM realize that this method of archiving also fits perfectly into the disaster recovery module of TSM. A second copy of all the archived data is also produced that will be shipped off-site - something that most likely didn't occur when clients archived their own data locally.

In looking at user needs and requirements, we also grappled with the prospect of automating the movement of end users' least-accessed data to a secondary virtual online storage location. This would be hierarchical storage management, or HSM. At first glance, this seemed like the best solution from an end user standpoint.

However, IT groups also need to carefully evaluate some of the risks involved in long-term storage via HSM before they determine if it makes sense for their organization. While it's true that HSM does immediately manage storage issues, there are also long-term storage issues (six months to one year down the road) that need to be considered:

  • HSM would manage the disk space, but it still does not manage the data.
  • There are high administrative costs to support HSM
  • There is a capacity limitation in regards to keeping the data that has been HSM'd in an online form.
  • There is still a finite amount of space in a tape library.

All HSM does is merely propagate to another device the problem of running out of disk space. While HSM is a viable tool for storage management, it is only one piece of the total data management package.

A delicate balancing act for IT, the needs of end users and your organization

For the items we've discussed in this story and sidebar (archiving through the help desk, end-user archiving, automated archiving, HSM and quotas), it may take any combination of one, two, three, four and/or all five of these tools. Each company is different and should be looked at as such.

Outside of communicating to end users and education, how can you best achieve success in storage management? Gaining buy-in from upper management makes this process considerably easier. Upper management in many companies often sees IT as an expense to the business as opposed to enhancing the business. Sometimes, it is not easy to be heard at that level.

Perhaps, if a business case were developed identifying down-side risks and upside benefits, it might help. Perhaps you could identify to management such items as:

  • The dollar impact of lost work due to system downtime or running out of disk space
  • Hardware savings involved in not buying CD burners and zip drives
  • The dollar impact of lost work due to the end user managing secondary media
  • The cost of data loss when local hardware media is in use for archiving and becomes lost

Once upper management buy-in is there, however, it all flows down the management chain.

About the author: John Vignoni is a Unix System Administrator with Lockheed Martin, Enterprise Information Systems. He has a foundation in application programming on the mainframe and Unix systems environments.

Dig Deeper on Data storage strategy

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.