This article can also be found in the Premium Editorial Download "Storage magazine: Backup overhaul: From a mainframe to an open-systems environment."
Download it now to read this article plus other related content.
Determining when to move to the next level is never easy, and it's no different with storage arrays and switches. If money didn't matter, oversizing the infrastructure could be one approach, but tight IT budgets and a mandate to do more with less make this a no-go for most environments. Fortunately, there's a systematic approach to right-sizing your storage infrastructure and it starts with planning, designing and architecting your storage landscape. An analysis of the hosts/servers and applications that connect to arrays and switches will provide the information you need to determine the average and peak bandwidth each server requires; this, in turn, will enable you to determine the number of ports and bandwidth per port your switches/arrays need to provide.
This analysis must include which servers can accept some level of congestion and those that absolutely need to operate at full line speed. Armed with this data, you can assign critical as well as storage-intense servers like database servers, transactional application servers and backup servers to full line-rate ports and have less storage-dependent servers like DNS, fax and other auxiliary application servers leverage oversubscribed ports. The more you take advantage of oversubscription, the more difficult this exercise will become.
Once the storage system is in production, the single most important tool for tuning or adding ports and capacity is performance monitoring. Nothing beats actual performance, utilization and latency data to isolate and remedy bottlenecks. For oversubscribed ports, the monitoring data will provide a clear picture of how often a port will reach maximum utilization. You'll be able to identify the servers causing these spikes, while the I/O data determines whether to move the server to a different port. User feedback is a crucial element, especially in environments with oversubscription. A storage manager may deem a port that reaches maximum utilization a few times a day acceptable, but it may not be acceptable to a business user performing a critical task.
For high-end end arrays, the guiding principle is performance and scalability and, naturally, complexity increases. High-end arrays are very modular, and the number of host ports and back-end ports is determined by business requirements. Ports are added on an as-needed basis. For instance, if you need additional disk enclosures, you simply add back-end disk directors or cards to support the additional spindles. Similarly, if you need more ports and bandwidth, just add front-end host directors or cards.
Unlike midsized arrays, the equilibrium of a balanced system can be easily broken. For instance, if you add host ports without beefing up the back end by adding more ports and controllers, the array back end will be unable to keep up with the increased host load. Performance and utilization monitoring are recommended for any size array, but it's imperative to ensure that high-end arrays stay tuned. Some of the elements to monitor include front-end host ports for fan-out ratios and host traffic, the cache subsystem, back-end disk processors, back-end I/O paths, and array or RAID processors.
High-end arrays can easily scale beyond 100 ports. To support this level of scalability, array vendors have implemented advanced array architectures. EMC's Direct Matrix Architecture in its Symmetrix array family employs a point-to-point design that directly connects front-end channels with array cache and back-end channels, eliminating any elements that could introduce delay. Array capacity is scaled by simply adding channel directors for host communication, back-end disk directors (two to eight, depending on the model, with eight ports per director), and global memory directors for I/O delivery from hosts to disk directors.
Unlike EMC, the Hitachi Data Systems (HDS) Corp. TagmaStore Universal Storage Platform (USP) implements a parallel crossbar switch with its Universal Star Network architecture, which connects front-end ports to back-end controllers and disks to cache. Similar to the EMC Symmetrix array, port count is scaled by simply adding front-end directors and back-end directors, supporting up to 192 FC ports, 96 ESCON ports or 96 FICON ports.
IBM's TotalStorage DS8000 array family--formerly known as Shark storage servers--takes yet another architectural approach, leveraging clusters of IBM System p servers that act as storage controllers where two (DS8100) or four (DS8300) of these systems are connected via a high-speed internal bus (RIO-G), supporting a total of 64 (DS8100) or 128 (DS8300) ports.
This was first published in April 2007