|SPICE: Core/edge SAN math|
In contrast to a SAN island or collocation SAN, the core/edge SAN offers the highest degree of availability for the most critical SAN component: the disk frame. While a server may be highly critical to the application manager, it's not nearly as critical as the disk frame that's shared among many servers. Therefore, one precept of the core/edge topology is that all storage devices should be located on a highly available directorclass or core switch. The servers can be located on the low-priced ports of the edge switches.
The disk frame is usually the most expensive SAN component and, under ideal conditions, it should be fully utilized before a new frame is purchased. A core/edge SAN design goal is to devise an architecture that allows the disk frame to become the bottleneck in the SAN, rather than the network between the frame and its contingent of servers. To achieve this objective, the bandwidth between the disk and core switch should be highly utilized. If there is a 2Gb/sec pipe between a disk port and the core switch, servers can be added until their average utilization approaches that 2Gb/sec limit. The number of servers required to reach this limit is usually referred to as the fan-out ratio of disk to servers.
The building block of the core/edge SAN is the core switch, which is populated with ISL and storage connections. The ISL connects the core switch to the edge switch attached to the servers.
In a core/edge SAN, the bandwidth needed by average servers is easily determined as the bandwidth of a pipe, usually 2Gb/sec, divided by the PI (the number of servers per storage port or the number of servers per ISL). If the PI has been accurately determined and a particular server becomes a bandwidth hog, then only that server needs to be moved. Because the architecture is designed to handle the clearly defined average performance of a server on an edge switch, the "hog" server would need to be migrated onto the core switch or the architecture is put at risk. When the PI is clearly defined, then the process for dealing with anomalies is easier to address; servers can be shuffled around to meet their capacity needs.
It's better to underestimate the PI than overestimate it. For example, a PI of seven means that two 24-port edge switches could support only 42 servers with six ISLs. If the PI had been estimated at 11, then those same two edge switches would support 44 servers with only four ISLs. If a PI of 11 was used, but a PI of seven was more accurate, then the ISLs between the edge and core would become the bottleneck and the disk could never be fully utilized. Conversely, if a PI of seven had been used, there would be excess capacity that could be used after the successful implementation of the SAN. Most importantly, lowering the PI to a safely attainable goal results in a core/edge design that contains room for additional hardware; raising it too high will result in an underfunded project.
The PI also determines overall capacity of the SAN (see "Applying the PI to Brocade, Cisco and McData switches"). For any configuration of core and edge switches, the PI sets how many servers one core switch can support. After that limit is reached, a new core switch is required. Regardless of size, the core switch will be roughly evenly divided between ports dedicated to storage devices and ISLconnections (see "SPICE: Core/edge SAN math"). Some core ports will need to be reserved for servers with higher bandwidth needs.
If you're migrating from an island or collocation SAN topology into a core/edge design, you can gauge the PI requirements of the core/edge SAN by gathering metrics from the SAN servers on the legacy environment. The PI gathered from the legacy SAN will be lower than what can be achieved once the storage devices are consolidated and can be more widely shared.
Once you have determined the PI that reflects your applications and their storage usage, you can quickly set a number of topology parameters, such as determining how many edge switches you'll need for your core/edge SAN, or how many servers a core switch can support.
This was first published in November 2004