This article can also be found in the Premium Editorial Download "Storage magazine: Using two midrange backup apps at once."
Download it now to read this article plus other related content.
While it's possible to back up this host in less than 24 hours, a general rule of thumb when examining clients is that any system with more than 500GB of data is a potential SAN client. Rather than backing up over the IP network, a SAN client is configured to back up across the SAN fabric directly to the backup media. Leveraging SAN-based backups for large clients eliminates the IP network and media servers as potential bottlenecks. While it may increase the CPU load on the client servers, most users are willing to sacrifice that additional CPU overhead for faster backups and, more importantly, faster restores. In this particular case, the firm installed an additional host bus adapter (HBA) in this problem client, and shortened the backup time from 20 hours to two. This meant restore times for this host were dramatically reduced.
Once you've identified and eliminated bottlenecks at the client level, it's time to focus on the actual backup infrastructure, starting with the backup servers (including master and media servers). I'm often asked how much memory and how many CPUs these servers need. While CPU and memory are critical components of the servers, they're rarely the source of bottlenecks (that said, if you measure RAM in megabytes or CPUs in MHz, it's time for a change). More often, the bottlenecks can be traced to the number and speed of the network connections.
At the front end
| of the server we have the IP connections. Insufficient IP connectivity is the most common bottleneck in the backup infrastructure. Many customers look at media servers as just another piece of hardware. Most often, they'll purchase a standard build that may include a dual-ported gigabit NIC. While this may be sufficient for most production servers, it can create serious throughput issues in a backup environment.
For example, if you've remediated your biggest client-side bottlenecks by implementing a dedicated backup VLAN with gigabit connections, it's possible that just two backup clients could consume all the network bandwidth into the media server. Admittedly, these would have to be heavy-duty client machines to max out the incoming data pipe, but it's easy to imagine a scenario where 10 clients, all with gigabit connections, back up to a single media server with two gigabit connections. In that scenario, the clients will be competing for bandwidth at the media server level. In environments with hundreds or even thousands of backup clients, you might need several dozen media servers with multiple trunked network cards just to provide enough network bandwidth to meet the backup window. Alternatively, as 10Gb/sec networks become more common, one of the first systems to upgrade would be the media servers. Upgrading the front-end connectivity for these systems shifts the bottleneck away from the IP network.
After resolving the end-to-end IP network bottlenecks, it's time to examine the backup targets and connections. Everyone's talking about backup-to-disk technology, but tape still plays a critical part in most backup environments. While many environments hope to go tapeless in the near future, many struggle to meet this goal due to extended retention times or tremendous amounts of backup data.
This was first published in September 2008