Managing and protecting all enterprise data


New directions for switches

With all of the recent acquisitions and new partnerships forming this year, finding the right Fibre Channel switch for you is even more confusing. This article will help you chose the right one.

Verify switch vendor claims
Rule No. 1: Test, test and test some more. Unless it's a one-vendor system, every storage environment is different. Vendors can certify their products with major storage products, but they can't possibly check every possible configuration with a storage area network (SAN). In other words, you're responsible for determining if a vendor's Fibre Channel (FC) switch works as advertised.
McData Corp., Broomfield, CO, continues to advertise that its directors support five-nines of availability. Yet some field engineers report that the McData directors they support lock up under certain conditions.
Brocade Communications Systems, in San Jose, CA, hasn't escaped user dissatisfaction either. Some users complain about the number of inter-switch links (ISLs) required to tie together multiple Brocade departmental switches, the time required to perform firmware upgrades on multiple switches and they also express concern about what they perceive as Brocade's initial failings in the high end of the market. Nancy Marrone-Hurley, a senior analyst at the Enterprise Storage Group, in Milford, MA, goes so far as to say that unless Brocade responds at the high end of the FC market, the high-end market will be dominated by McData and Cisco.
Computer Network Technology Corp. (CNT), Minneapolis, MN, appears to be a mystery wrapped up in an enigma to end users. Users at a breakout session at the Storage Decisions conference in Chicago rated it as fourth among switch vendors and considered Cisco a more acceptable choice.
Inrange Technologies, Lumberton, NJ. Former employees of Inrange say that part of the reason they left Inrange is that they believe the company wasn't delivering on the software feature functionality that they internally were told existed in the product. They found out after they sold the product the functionality wasn't there.
The bottom line is that all of the switch vendors have some skeletons in their closets. It's premature to look to any one FC switch vendor to provide all of the answers or expect their existing product line to work flawlessly in every environment. For now, users should view current purchases as more tactical than strategic.

Security, storage and transport services are establishing residence within the fabric either as standalone appliances or in the switch. Hardware and software vendors with no previous presence in the network hope to establish their service as the default solution in the fabric.

As these interlopers move in, how do the dominant Fibre Channel (FC) switch vendors respond? And more importantly, how do their new switches fit into your storage environment? 2003 was the year of acquisitions and alliances for switch vendors as they fought to gain a technical advantage against their competitors. McData Corp., in Broomfield, CO, bought Sanera Systems Inc. and Nishan Systems; Brocade Communications Systems Inc., based in San Jose, CA, purchased Rhapsody Networks and Cisco Systems took over Andiamo Systems. Meanwhile, EMC Corp. and Veritas Corp. announced that they will form strategic alliances with Cisco and Brocade to port their storage management software to the fabric, but provided little details on when and how.

Most importantly, how should users respond to all of these announcements? One approach is to throw caution to the wind and begin deploying these new fabric technologies. The more cautious approach in selecting a new switch is to rely on more traditional FC measurements of port count, interoperability, reliability and high availability when making a selection, ignoring, at least for the moment, the hoopla surrounding these dramatic changes being brought to the FC fabric.

For now, storage area network (SAN) admins looking to upgrade switches need to check out new FC transport services such as 4Gb and 10Gb FC speeds, virtual SANs (VSANs) and hard partitioning. They also need to pressure vendors for better solutions to their inter-switch link (ISL) issues. Security administrators should evaluate the new security risks that FC switches introduce into the organization; storage admins should start to weigh in on fabric-based virtualization, volume management and other storage services beginning to appear in the fabric.

FC transport services
If you're only looking at switches for transport services, 1Gb and 2Gb FC continues to be a mainstay for connecting servers to storage. FC switches and directors provide a highly available, scalable and reliable method for block I/O storage networking connectivity. (For more information, see "Switch selection criteria"). Furthermore, these devices offer a growing diversity of hardware choices. Switches appear in configurations as small as four ports, such as the one offered by McData in its Sphereon 4300, while directors are available in port counts as high as 256 from vendors such as Inrange, Sandial and Sanera Systems (see "High-end storage switches").

These FC switches and directors offer benefits that uniquely qualify them over other networking protocols for moving block I/O between servers and external storage arrays. In addition to FC's ability to efficiently move large block I/O, these switches have expanded their legacy transport services such as data integrity, routing and trunking to offer enhanced functionality such as 4Gb and 10Gb FC, virtual and partitioned SANs, improved security and better vendor interoperability.

One nuance of the FC protocol that helps to ensure data integrity and sets it apart from other networking protocols is the requirement for all data frames to arrive in sequence. Unlike TCP/IP, which can reassemble data packets of information regardless of the order in which they arrive at their destination, the FC protocol requires that data frames arrive in the same order in which they were sent. This creates difficulties for performing advanced functions between different switches, specifically routing and trunking between switches from the same and different vendors.

How the routing and trunking functions get handled by the different switch vendors continues to be a bone of contention between them. While all vendors support the current FC standard of connectivity between switches using ISLs that assign ISLs to servers based upon a round-robin distribution methodology, Brocade's switches users treat up to four separate ISLs as one logical path.

This feature offers significant advantages over the current FC standard. First, this capability treats four physical ISLs as one logical ISL. Second, it allows for the gathering of meaningful historical performance information because under the current FC standard, there's no reliable way of knowing which ISL any particular server is using. Under the current standard, every time a server logs off and back onto a FC fabric, the server may be reassigned a different ISL using the existing round-robin methodology when multiple ISLs are in place.

Finally, Brocade's trunking reduces the number of ISLs needed between switches. The trunking function can aggregate the bandwidth and utilization of their links, treating four 2Gb ISLs as one logical 8Gb pipe. Under the current FC standard, four separate ISLs are exactly that, four 2Gb pipes with a maximum capacity of 2Gb each that lacks a nondestructive mechanism to allow an overloaded ISL to send some of its FC traffic down another lesser-used ISL.

The dispute between existing vendors arises not in the value of this technology, but in its implementation. Brocade deploys this technology using its proprietary hardware ASIC solution. While users have the option to turn this feature off on Brocade gear and connect ISLs to another vendor's switch, the feature is currently an all-or-nothing option on Brocade switches. So unless users exclusively use Brocade switches and directors, they lose this important value add.

A larger issue grating on users emerges in terms of how vendors choose to deploy ISLs. For those users who need to link switches together, the only way to do so now is by consuming some of their existing ports for ISLs, as opposed to having these switches offering back-end ports that provide for this functionality. While annoying, users bristle when vendors tell them there is no demand for this service. At the fall Storage Decisions 2003 conference, Sanjay Mandloi, a vice president with LabMorgan Technology Solutions of JP Morgan Chase, stated that both JP Morgan and Merrill Lynch are asking vendors to provide this functionality.

Pros and cons of
some major switch vendors
Brocade Communications Systems brings extensive experience in midrange and departmental switching and offers one the most secure and tested firmware/OS of major Fibre Channel (FC) vendors for functions such as security and routing. What to watch for in the long term, is its ability to complete the integration of Rhapsody Networks as well as other storage services into its portfolio of products and overcome its perceived failings in the high end of the market with its SilkWorm 12000. For the short term, one should look to Brocade to provide a more secure fabric and a higher level of compatibility than its competitors and remain the preferred solution in the midrange and departmental spaces.
Cisco brings a depth of networking experience and talent to the table. However, some of the dynamics of FC and IP networks differ both in terms of expectation and design. Look for Cisco to start to gain market share, get the kinks out of its product and put to rest the swirling rumors about the reported failings of its MDS 9000 directors. For the short term, treat Cisco's products as the new kids on the block and require the company to prove itself in this space.
CNT has years of networking experience, and with its acquisition of Inrange now has a highly available, high port count director to complement its WAN offerings. Watch to see if they gain market share and improve its perception in the marketplace. In the short term, continue to look to them to provide storage network connectivity over large distances with its UltraNet line of products and as a viable FC director alternative with its FC/9000 line of products.
McData brings some of the most extensive and timetested data center experience to the FC director market and is now moving into the midrange and departmental space with its Sphereon line of products. McData's challenge over the next 12 to 18 months will be to integrate its recent purchases of Nishan Systems and Sanera Systems while improving the manageability and intelligence of its microcode. For the time being, look to its Intrepid 6000 series of directors as the preferred solution for enterprise-class high port count FC solutions.
QLogic is the only one of the major FC switch vendors that offers FC host bus adapters (HBAs) as well. With the introduction of its SANbox2-64, they now also offer a 64 port FC switch solution that competes in the enterprise market. QLogic's longer-term strategy requires them to capitalize on the market acceptance of its HBAs as a way to introduce its FC switches into existing environments. In the short term, expect QLogic to remain a niche FC switch player.

Faster speeds
Yet for all the value a logical 8Gb channel comprised of four physical 2Gb ISLs may offer users, the approved 4Gb and forthcoming 10Gb FC standard may make this trunking functionality obsolete, or at least minimize its current value. With the future incorporation of these standards into the switches, one single 10Gb ISL between any two vendor's switches supporting the 10Gb standard would offer more bandwidth than four aggregated 2Gb links between two Brocade switches and use less ports.

QLogic Corp., Aliso Viejo, CA, is the primary switch vendor advocating the use of 4Gb FC and was recently successful in campaigning the FC standards board to approve this standard. While the 4Gb doubles the current 2Gb FC standard, its primary value to users comes in the ability to deploy this technology without requiring a major infrastructure technology upgrade. Because the new 4Gb standard will be backward compatible with existing 1Gb and 2Gb HBAs used in organizations today, users can expect its existing FC infrastructure to serve them for the foreseeable future without an upgrade.

While the proposed standard will require a technology upgrade, the 10Gb standard holds particular appeal for users needing simple and fast switch interconnects and high-speed connectivity to storage arrays and tape drives. Sanera Systems became the first vendor to offer and support 10Gb functionality on its DS10000 256 port director. Of the 256 ports, the director supports up to 64 ports for 10Gb functionality. Sanera Systems' CEO Patrick Harr says they are able to offer this functionality already because even though the 10Gb standard isn't ratified, the physical layer is already defined. However, unless connecting two of Sanera's DS10000s, this feature will be of limited or no benefit because no other vendor currently offers 10Gb connectivity.

10Gb FC also opens up new ways to think about disaster recovery (DR) as it extends the distance between two sites for DR. Sanera System's Harr points out that the maximum distance for 1Gb FC is about 27 km. 10Gb extends an organizations ability to maintain synchronous copies of data to 270 km because the latency issues around synchronous writes are extended due to the faster 10Gb FC speed.

Users looking for 10Gb anytime soon, however, will have to wait. Mike Witkowski, CTO of Maxxan Systems Inc., says that while he expects 4Gb and 10Gb in the second quarter of 2004, and availability largely depends on how soon interoperability testing with the major OEMs can be completed. He also points out that with the recent ratification of 4Gb, QLogic is accelerating 4Gb testing over 10Gb, which may further delay the general release of the 10Gb standard.

Yet today's users are beginning to look for more than just ways to use what they have at faster speeds. They're once again beginning to turn their attention to new features and options that lower the overall cost of business while simplifying their lives. Recent switch enhancements and acquisitions show that vendors are listening to this change in attitude.

Virtual or partitioned SANs
As a way for users to tie their existing SAN fabrics into a central core switch and protect the integrity of their fabrics, Cisco reintroduced the VSAN concept back into the FC market as part of the introduction of its MDS 9000 in early 2003. Originally proposed by Vixel Corp., Bothell, WA, in the early arbitrated loop days of SANs, VSANs enable different fabrics to exist on one switch or director that are oblivious to the existence of other fabrics on the same switch. However, this concept has gotten little traction to date because Cisco is still in the process of breaking into the FC market and users are still primarily at the early stages of figuring out how to best implement this option.

The concept of a VSAN being solely offered by Cisco's MDS 9000 will now face a stiff challenge from McData as a result of McData's recent Sanera Systems acquisition for a couple of reasons. First, Sanera Systems' DS10000's underlying code introduces hard partitioning into the FC environment. This differs from VSANs in one important way. Like Cisco's MDS 9000, it allows existing FC fabrics from different vendors to connect into one central switch. Unlike the MDS 9000, each partition on the DS10000 can maintain a separate microcode level and be upgraded separately, thereby ensuring compliance and interoperability with the fabric which it serves regardless of the vendor. Second, with McData already so well entrenched in many data centers, and because the DS10000 appears to offer a solid if not better alternative to Cisco's solution for some environments, it will likely further slow Cisco's plans to establish a presence in the high end FC space. This, of course, assumes McData can motivate its partners to certify the DS10000 in existing SAN fabric configurations in a timely manner because many of these same partners also have similar partnership agreements with Cisco.

Despite the promised benefits, Tom Clark, Nishan Systems director of technical marketing, sees VSANs as a step backward for the FC switch and completely counterproductive to the concept of utility storage. He says that VSANs can't share storage resources with each other. For instance, if a tape drive is assigned to one VSAN, the tape drive in a specific VSAN is no longer available as a resource to other VSANs on that same switch.

Emphasis on security
Switch vendors are starting to offer more security features and connection options for remote or disparate SANs. The recent acquisition by McData of Nishan Systems and their IPS Storage Switches adds to McData's portfolio of products the ability to connect disparate SANs using IP or FC while maintaining the integrity of each fabric.

Computer Network Technology Corp. (CNT), based in Minneapolis, MN, and Inrange Technologies Corp., Lumberton, NJ, also recently became one company when CNT acquired Inrange in May of 2003. By acquiring Inrange, CNT obtained Inrange's line of FC/9000 directors that scale from 24 to 256 ports to compliment CNT's Ultranet product line that offer SAN, MAN and WAN connectivity. Cisco's acquisition of Andiamo paralleled CNT's acquisition of Inrange in terms of bringing a high FC port count director into their product mix. Brocade offers ports on its SilkWorm 12000 switch that can be used to extend and connect its products to different SANs.

Even the decades-old SCSI protocol may be getting a new lease on life thanks to Crossroads Systems Inc., in Austin, TX Its ServerAttach SA40 grants legacy midrange SCSI-attached servers, tape drives and storage arrays the ability to connect to FC SANs. The SA40 functions as a SCSI to FC protocol converter and can support up to four 2Gb FC ports and eight SCSI ports.

This technology preserves and extends the life of existing SCSI implementations while also offering a FC connectivity option to those operating systems which don't natively support FC, such as AS-400s and older versions of more popular operating systems such as Windows NT 4.0. Now these servers can gain some of the benefits found in FC SANs such as shared DASD and tape resources without requiring the deployment and added expense of FC HBAs in these servers.

A fully configured SA40 costs about $20,000 or $2,000 per port which may be a bit high to gain this functionality for SCSI connectivity. Users will need to weigh this cost to connect their SCSI devices to FC SANs vs. migrating them to FC, which in some cases isn't even an option due to the age of some of these technologies.

Sandial Systems, a startup based in Portsmouth, NH, is on course to deliver yet another enterprise class director. In addition to supporting technologies such as 256 ports, virtualization, and V-SANs, Sandial purports to offer the industry's first storage network backbone. Their switch features a time division multiplexing engine that helps users overcome the current oversubscription limitations of ISLs used in today's SAN fabrics and enable them to build more scalable FC storage networks.

Connecting these disparate SANs with different protocols also raises the increased possibility for breaches in security. While multiple mechanisms are in place to protect IP networks, FC networks often lack the same sort of attention. Right now, the biggest levels of security around FC networks are the general ignorance about FC protocols and the fact that the attached servers are often situated in physically secure locations accessed by a limited number of personnel.

Yet as SANs become more interconnected with more protocols in more accessible locations, the likelihood increases that network security will be compromised. For instance, in environments that use the world-wide names (WWNs) assigned to the server host bus adapter (HBA) to do zoning as opposed to port zoning, if someone changes the WWN to match that of another server on the network, that server may then have access to the other server's storage on the storage network.

Most vendors have some precautions in place to prevent this. McData offers the ability to map a specific WWN to a specific port, so if a different WWN logs into that port, it locks the new WWN out. However, this approach is akin to issuing one key to one person for a room and no one but that person with that key can enter the room. To allow another person to enter the room, you need to take away the key from the person who has access, reissue a new key and re-key the door. While highly effective, it's probably overkill as it becomes too labor-intensive for any environment but the most static or security conscious and, as a result, discourages most users from implementing it.

Brocade offers users the ability to place a token on each server so that each time a server logs onto a FC fabric, its Fabric OS authenticates the server's token. However, the license keys and authentication are currently only issued by Brocade, so if any security breach does occur, who is liable for the breach becomes a real issue because Brocade hedges on assuming this liability.

Yet an informal sampling of users at this year's Storage Decisions 2003 still minimize the security threats FC technology introduces into their environment. With many FC deployments supported by trained personnel in closed environments, the risk for now appears mostly limited to ignorance or human error and is outweighed by the cost to implement additional security measures.

The "V" word
One word that just won't go away is virtualization. Michael Passe, a senior systems engineer with Beth Israel Deaconess Medical Center, believes that virtualization will move to the switch, he just doesn't know when. So in the meantime, vendors are working overtime to rebrand virtualization technology in terminology that resonates better with end users. Vendors now refer to it as network-based volume management, provisioning or a storage controller, anything but the dreaded V word. Despite their efforts not to play up this feature, every vendor with the exception of CNT now either offers this feature natively in their switch OS, has partnered with someone such as EMC or Veritas to bring their technology into the network or has purchased the technology.

Vendors that offer it natively in the switch OS show up primarily as the new kids on the block, be it Candera, Maranti Networks, Maxxan or Sanera. Candera offers its Network Storage Controller that acts like an appliance on the network that manages storage and works with the existing network infrastructure. Their approach complements, rather than replaces the existing infrastructure, akin to how IBM Corp.'s Storage Volume Controller (SVC), DataCore's SANsymphony, Softek's Provisioner and FalconStor's IPStor products work.

High-end storage switches
**Acquired by McData, *Assumes 1Gb, +Assumes 2Gb

Maranti, Maxxan and Sanera all elected to incorporate their technology into their high-port count directors. Again, this solution can either complement or replace an existing SAN depending on what problem you're trying to solve. If you have a fairly static environment and one group of individuals handles all zoning and storage allocations, you may want to look at replacing your existing technology with their solution. However, users remain leery when it comes to storage and replacing their existing solution with a new one that promises to solve all of their problems. Here's where a solution such as is offered by Candera may make more sense because it allows for an organization to gradually step into this environment.

McData's acquisition of Sanera Systems allows them to join the ranks of Brocade and Large vendors with a storage virtualization solution. Cisco obtained technology similar to Sanera Systems when they acquired Andiamo. While Brocade, by acquiring Rhapsody Networks, obtained a solution that more closely aligns with Candera.

So as the rush to put down roots in the fabric continues, existing FC switches will remain a mainstay in the enterprise. The future promises to be pretty much like the present: new players come out of stealth mode, and the established switch companies continue to form more alliances and tighter integration with existing storage players. However, all this frantic activity will not overshadow the critical functions FC switches perform in the enterprise today. In the longer term, of course, major switch vendors see the need to adapt to, merge with or acquire their new neighbors in the fabric or risk being replaced by these new fabric inhabitants.

Article 10 of 16

Dig Deeper on SAN technology and arrays

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.

Get More Storage

Access to all of our back issues View All