How do we significantly reduce the amount of bandwidth required for incremental backups? Many vendors have attempted to solve this through a technique often referred to as delta block or block-level incremental backups. In this method when a file changes in any way, (e.g., adding one line to your resume) the file's modification time changes. A typical backup and recovery system would then back up that entire file during an incremental backup - even though only a small portion of that file has changed. A block-level incremental backup would also recognize that the file has changed -- based on its modification time -- but would then determine exactly which blocks in the file have changed, and then back up only those blocks. This technique significantly reduces the amount of data that needs to be transferred to the backup server, as well as the amount of data that must be stored on disk on the backup server. Clearly, combining block-level incremental backups with an incremental-forever backup methodology makes backing up to disk even more important.
Some products perform an additional step aimed at reducing the amount of data that must be transferred during a backup. Ask yourself a question: How many copies of command.com need to be backed up? If a given file has already been backed up from another client, wouldn't it make sense to note that you found another occurrence of the file? It makes sense to only back up each file once, regardless of how many systems where that file resides. This technique is referred to as redundant file elimination.
There are several commercial products designed specifically to back up desktops and laptops. A directory of them may be found here.
A new company called Avamar Technologies, Irvine, CA, combines the functionality provided by block-level incremental backups and redundant file into a single concept -- redundant block elimination. Their product, Axion, finds logical sequences in files, objects and databases and stores each unique logical sequence only once per Axion backup system. As a result, edited files, copied attachments, applications replicated across systems and even daily changing databases require an extremely small amount of backup storage. While this particular article is about desktop and laptop backups, Avamar has applied this unique concept to the world of server backup as well.
What if you're only one user or a small company with a limited budget? Although the solutions for such environments may not have equivalent functionality to the products described above, there are solutions available -- most of them in the form of replication.
This article originally appeared in Storage magazine.
About the author
W. Curtis Preston is the president of The Storage Group. He is the author of Unix Backup and Recovery and Using SANs and NAS.