Big files create big backup issues


This article can also be found in the Premium Editorial Download "Storage magazine: How to plan for a disaster before a software upgrade."

Download it now to read this article plus other related content.

Another Norton Healthcare unit needed a different fix, however. It uses an EMC Corp. Celerra to house personal home directories and departmental shares, amounting to 5.5 million files and 2.5TB of data. "On this server, we split the backups into sections to shrink the backup window," says O'Mahoney. "Incremental backups now take under an hour, and we have recently received a recommendation from EMC to increase parallelism [to run several backup processes at the same time], so we hope to reduce the full backup time from the eight hours it takes today."

When faced with a massive number of files, splitting them up and running multiple backup jobs in parallel can be a big help if your client and backup server can handle the load. Bill Mote, systems engineer at Cincinnati-based Making Everlasting Memories L.L.C., saw huge benefits when parallelizing backup on a server containing millions of image files. "We split the directory tree into 10 smaller ones to improve performance, manageability and scalability for our application," says Mote. "Now we can point the backup application at a subset of the total data and run multiple jobs in parallel."

When using IBM's TSM to back up a Windows client, a special option called journaling is available. "TSM normally examines each file and compares it to the database, creating a list of files to be backed up," explains John Haight, master

Requires Free Membership to View

consultant at Forsythe Solutions Group. "Although this is a very efficient process in general, when presented with millions of files it can take hours. In the case of journaling, the client keeps track of which files have changed and notifies the backup server, rather than scanning the whole file system." When journaling is enabled, the results can be dramatic. "The time required to scan our imaging system dropped from over 24 hours to just a few minutes when we tried TSM journaling," says Mote at Making Everlasting Memories.

But there are some issues with this technology. "Files that are deleted aren't cleaned up with TSM journaling, so a normal incremental is needed occasionally to clean this up," says Haight. Some experts also voice concern that the journal might be deleted in some instances, forcing a complete file system scan. And the technology is limited to a single OS and backup app.

This was first published in May 2008

There are Comments. Add yours.

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: