Published: 12 Aug 2007
Brocade's 48000 Director and Cisco's MDS 9513 Multilayer Director offer different paths to storage services and...
consolidation options. Which company offers the best director for your storage environment?
SAN consolidations, blade servers, virtual servers and storage, data encryption and requirements for more secure fabric services are prompting enterprises to bring more application services into their SAN fabric. Fibre Channel (FC) switch vendors Brocade Communications Systems Inc. and Cisco Systems Inc. are adding new features to their FC directors' core operating systems or on specialized FC director blades that support these new applications.
For Brocade, the future of its FC directors clearly lies with its 48000 Director model. Though Brocade plans to continue support for its other FC director models (see "The other Fibre Channel directors," below) into the foreseeable future, its 48000 Director is clearly the flagship model for new switch services. "The 48000 will sit at the core," says Doug Ingraham, Brocade's senior director of SAN product management. Application blades such as the FR4-18i Director Blade, FC4-16IP Director Blade and the FA4-18 Application Blade are only available for the 48000, and Brocade doesn't plan to port these blades or services to its other FC directors.
The other Fibre Channel directors
In addition to Brocade Communications Systems Inc.'s 48000 Director, its Mi10K Director and M6140 Director may be a good fit for some organizations. Here are some examples:
- Complete separation of FC and FICON ports. Mi10K's hard-partitioning feature allows complete separation of these environments, so day-to-day management like firmware upgrades, zoning changes and user management can occur on one partition with minimal or no effect on the other.
- Regulatory compliance. Some companies have regulatory requirements that require some of their data to be completely isolated. Hard partitioning allows companies to meet these stringent data-separation requirements.
- SAN extension over large distances. The Mi10K supports 10Gb/sec interfaces and the highest number of buffer credits (1,133) per port of any Brocade FC director, which enable high-speed, long-distance links.
- Widely deployed. The M6140 is the most widely deployed FC director on the market, so most of its issues are now known and have workarounds available.
- Simpler to manage. It lacks many of the advanced features that FC directors like the Brocade 48000 Director and the Cisco MDS 9513 Multilayer Director support, but it's easier to understand and manage.
- Fewest ports per blade. Allows companies to grow the M6140 in increments as small as four ports.
- Medium-sized companies. Can serve users who want a platform that can meet their FC and FICON requirements but who don't anticipate growing beyond 140 ports.
Cisco is further down the road in offering fabric services in its MDS 9513 Multilayer Director SAN-OS, and on its Multiprotocol Services (MPS-18/4) and Storage Services Module (SSM) blades. (The SAN-OS term is commonly used across all Cisco FC switches and directors. However, there are differences in SAN-OS functionality on the various models, so we use the term MDS 9513 SAN-OS throughout this article.) However, services such as continuous data protection, replication, volume management, virtualization as well as Cisco's new Data Mobility Manager (DMM) are only available if the optional SSM blade is installed on a MDS 9500 Series Multilayer Director. (Encryption services are supported on Cisco's MDS 9222i Multilayer Fabric Switch or MPS-18/4 director blade.)
The directors from Brocade and Cisco are at different stages in their ability to help companies consolidate, control, and share their local and remote FC SAN resources (see "Key considerations to selecting a Fibre Channel director," PDF). For companies with highly distributed management environments that don't want central control, Brocade's architecture lets users introduce fabric services that preserve local administrative control while sharing SAN resources. Companies that need to centralize and standardize their FC infrastructure across the enterprise will be better served with Cisco's FC director architecture.
Consolidation and control
Consolidating remote SAN islands includes technical challenges and political issues about who will control the resources after the consolidation is complete. FC directors currently provide two methods to perform SAN consolidations: fabric mergers and fabric joins.
Brocade's 48000 Director and Cisco's MDS 9513 Multilayer Director allow companies to merge SAN islands into one large, logical fabric (see "Director FAQs"). But fabric mergers mean turning control over to one party--usually the department with the larger FC director. The merger also comes with countless technical intricacies. Verifying fabric settings and ensuring they're correct can exceed the time and risk thresholds of many companies. Rather than forcing companies to deal with these issues, Brocade and Cisco permit companies to join remote SAN islands to their large FC directors while keeping them logically separate. Users can share resources among virtual or logical SANs; if a tape drive or disk ports reside in one virtual SAN, servers in another virtual SAN can still access them.
That's where the similarities end. Cisco's virtual SAN (VSAN) technology is part of its native SAN-OS operating system found on its MDS 9500 Series Multilayer Directors. This permits FC switches, or directors of separate and potentially remote SAN islands, to connect to any FC port on the MDS 9500 and remain logically separate from other VSANs that exist on the MDS 9500. Brocade's logical SAN (LSAN) implementation is available only as part of its 48000 FR4-18i Director Blade or on its external 7500 SAN Router.
Which approach is better depends on who'll control what resources after the consolidation. Cisco's FC director architecture assumes companies will want to:
- Consolidate or connect remote SAN islands onto its MDS 9500 Series Multilayer Directors.
- Centralize the administration of user and administrator accounts in the FC SAN.
- Centralize the control and sharing of SAN resources across virtual and remote SANs.
With VSAN functionality a native part of the MDS 9513 SAN-OS, users don't have to buy specific blades or switches to obtain this functionality. Administrators may grow VSANs logically on a port-by-port basis instead of adding a new blade into a 48000 or introducing an FC switch into the fabric as Brocade requires. Using director blades or switches adds the step of moving the physical FC connections of resources (servers, tape drives and storage arrays) to the ports on the director blade or switch.
The MDS 9513 eliminates these steps by allowing admins to configure any MDS 9513 FC port as an "E_Port" and connect it to any vendor's remote SAN switch. VSANs are then created on the MDS 9513, which admins may define to just include the FC ports connected to the remote SAN island. Specific resources among VSANs are shared using Inter-VSAN zones (IVZs).
This is where the issue of control enters. The MDS 9500 Series Multilayer Director supports up to 1,024 VSANs in a single fabric and each VSAN supports the creation of individual admins and users. But sharing resources requires the creation of an IVZ by an admin with superuser privileges and access to all VSANs. The main concern with this approach is that superusers may create IVZs between VSANs without informing the individual VSAN admins. This allows someone external to the VSAN to control how and when specific VSAN resources are shared.
Doug Ingraham, Brocade Communications Systems Inc.'s senior director of SAN product management, and Deepak Munjal, Cisco's data center solutions senior marketing manager, answer some FAQs about their respective 48000 Director and MDS 9513 Multilayer Director.
Under what circumstances should users deploy application blades in Fibre Channel (FC) directors?
- Brocade: "Use a separate device to separately manage and administer applications based on their operational processes. Use a director blade when an open director slot is available; the throughput requirement is such that having a backplane bandwidth is an advantage over an ISL [interswitch link] trunk and the same administrator who manages the director also manages the application service."
- Cisco: "Since we support VSAN technology in the MDS 9513 SAN-OS, SAN virtualization is the primary play right now. EMC [Corp.]'s RecoverPoint and Incipient [Inc.]'s Network Storage Platform are examples of products taking advantage of the MDS advanced feature set."
- Analysis: Cisco currently provides a richer set of fabric services for application blades.
What's your FC director roadmap?
- Brocade: "The 48000 sits at the core and other FC directors and switches fan into it. Going forward, McData's EFCM [Enterprise Fabric Connectivity Manager] management software will become the de facto management platform for all of our products. We have a major product upgrade planned for early 2008."
- Cisco: "We expect the trend toward integrating more fabric services into the MDS 9513 SAN-OS to continue. We will continue to integrate appliance functionality onto a blade and, once it evolves enough and the chips mature, move it to the core SAN-OS."
- Analysis: For Brocade to remain the leading FC switch vendor, it needs to knock the socks off data center managers with the next-generation 48000 release (in 2008) with new features and heightened integration with its M-Series FC directors.
What recommendations do you have for management of your FC director postconsolidation?
- Brocade: "There are two major areas in which customers need to manage SANs: proactive fault detection and zoning. Brocade SAN Health monitors SANs and provides comprehensive support postconsolidation, while Brocade provides five zoning options. We let customers select the zoning option that makes the most sense to them rather than forcing them down a particular path."
- Cisco: "Consolidate disparate SAN and storage groups into one unit. SAN and LAN teams should also begin discussions as we anticipate a similar convergence to occur between these two groups."
- Analysis: These answers reflect their respective architectures. Cisco wants everyone to do everything the same way, and Brocade gives users a choice of how they manage their data.
Preserving SAN autonomy
Brocade's director architecture assumes companies will want to consolidate FC ports and share resources among LSANs, but leave control in the hands of specific business units. This approach avoids some of the bureaucratic haggling that Cisco's consolidation approach may cause. Martin Skagen, Brocade's director and chief architect, office of the CTO, finds the benefits of consolidation and partitioning great, but they create an extremely strict change control environment. Once consolidation occurs, "the moons have to align to change anything," says Skagen.
To prevent this and to keep remote SANs separate, Brocade's 48000:
- Connects remote SAN islands to its 48000 through a 7500 SAN router or the FR4-18i blade
- Allows each LSAN to retain control of the creation of user and administrator accounts
- Allows each LSAN administrator to retain control of sharing of LSAN resources
Both the 7500 router and FR4-18i Director Blade support a special type of E_Port for connecting different fabrics called an EX_Port. This allows Brocade to introduce its Fibre Channel Routing (FCR) feature into these two devices that allow different LSANs to communicate with one another.
FCR creates a virtualized switch called a "translate domain," which imports all of the nodes of the edge fabric. Resources are then accessed between different edge fabrics by creating a zone name starting with "LSAN_" that contains the port WWN of the nodes that will access each other. Brocade's FCR recognizes these new LSAN zones in the respective LSANs, automatically creates the appropriate routes and then allows communication between selected nodes in the two zones.
Brocade more fully preserves admins' control in separate and remote SAN islands by eliminating the need for a superuser. Admins of each remote SAN island or logical SAN first create an LSAN zone containing the nodes to be shared between LSANs. If an LSAN zone is created in only one LSAN, members defined in the zone aren't shared across LSANs by the FCR. This allows firms to preserve existing political boundaries and gain consolidation benefits.
Cost and complexity
As more services are added to the fabric, costs and management complexity increase. Costs may rise because each new service requires the addition of the appropriate switch to the fabric or director blade to the FC director. Because each new service introduces additional hops into the fabric, it may be prudent to add only one or two fabric services at a time.
Deepak Munjal, Cisco's data center solutions senior marketing manager, says some of the different fabric services will eventually move off switches and blades and become part of the MDS 9513's SAN-OS. Specialized ASICs on every FC port will eventually take over the processing for each service and let users select which services they want to offer to each app on a port-by-port basis. Munjal says cost and demand will drive these changes, but "it is not realistic to do this [now] since the price of putting an ASIC on every port is still not affordable."
It took three to five years for encryption to move from an appliance to a chip that sits on a router port in Ethernet networks, says Munjal. "Fabric services will likely follow the same path," he adds.
The growth of blade servers creates a problem for FC directors: port preservation. A fully populated IBM Corp. BladeCenter Chassis can use up to 28 FC director ports assuming two paths; a Hewlett-Packard Co. BladeSystem c-Class can consume up to 32 FC director ports, again assuming redundant paths. In addition to using up ports, blade servers often can't use the bandwidth available to them on FC director ports.
To mitigate these issues, Brocade and Cisco offer blade switches that act as gateways from the blade servers to the FC director. Blade servers connect to these gateways as they normally would to an FC switch with an N_Port login. However, the switch connects to the FC director and presents itself as an N_Port or host ports instead of as an E_Port as FC switches normally do.
This feature allows the gateway to take advantage of the virtual N_Port ID Virtualization switch standard and re-present each blade server N_Port login as multiple virtual N_Port logins to the FC director. Once the virtual blade server N_Port is logged into the FC director, the FC director can treat it as a server logged into the SAN, including zoning it to specific ports and assigning qualities of service to traffic coming from that blade server.
But there are differences with how the two vendors implement their gateways, including which vendors' blade servers are supported. Brocade's blade server SAN switch supports blade server chassis from more vendors, including Dell, Fujitsu, HP, Hitachi, IBM, Intel and NEC. Cisco's Fibre Channel Blade Switch currently works only in HP and IBM blade server chassis.
The gateways also scale back on certain software features their switch OSes typically support. For instance, Cisco doesn't support IVZ on its Fibre Channel Blade Switch, while Brocade disables nearly all features its FC switches support such as ISL trunking, FC-AL and Brocade Fabric Manager. Disabling switch features such as ISL trunking happens because the switch presents itself to the FC director as an N_Port instead of an E_Port or EX_Port, so the FC director recognizes the gateway as a storage device and not an FC switch.
Changing this requires the introduction of a feature called F_Port trunking into FC SANs. F_Port trunking is similar to interswitch link trunks between switches except that it's designed to create trunks between storage devices, such as server FC host bus adapters and gateways, and the target device. Driven by technologies like blade servers and server virtualization, it allows FC traffic to go down different paths but still assure in-order delivery regardless of which path the traffic takes. However, "F_Port trunking does not exist in the [Brocade] 48000 today," says Mario Blandini, Brocade's director of product marketing.
Because Cisco has incorporated services like VSANs and IVZ into its SAN-OS, it leads Brocade in many respects. Brocade's resources remain split between developing the next-generation 48000 and supporting legacy McData products. Though Brocade's planned 2008 release of a new FC director may tilt the balance, it will depend on what new features are built into its new director. In the meantime, firms shouldn't deploy more than one or two fabric services at one time (due to cost and complexity) until more of these services move off switches and director blades and into the core OS of the FC director products.
For a detailed chart listing Brocade and Cisco Fibre Channel director features, go to https://searchstorage.techtarget.com/.