Published: 12 Jul 2004
Celerity HBA is a Storage Star
Hollywood movie production houses were among the early adopters of Fibre Channel (FC) storage area networks (SANs). Because they must move large volumes of data between editing stations to manipulate audio and video characteristics, applications often require the largest data pipe available. If you've noticed how quickly new DVDs hit the video store shelves, you can chalk it up to the increased productivity resulting from FC SANs.
Atto continues to increase its footprint in tinseltown by providing products in lockstep with Apple Computer Inc., the favorite platform of movie production houses. It's surprising that no other HBA vendor currently has a comparable product for this space. Apple sells a dual-ported, independent channel PCI adapter for connectivity to its XSAN line of products for $500, and Charismac Engineering Inc. has rewritten Emulex Corp. drivers to support a limited number of HBAs for Mac OS X, but that's it.
The Celerity FC-24XL is supported under Mac OS X 10.3.x, various versions of Windows and Red Hat Linux 8.x and 9.x.
Although backward compatible with PCI, the Celerity FC-24XL should be installed in a 64-bit PCI-X slot to achieve maximum throughput capabilities. If port density--not speed--is your main objective, then installing the card into a 64-bit PCI slot will work.
After installing the HBA into a Power Mac G5 server, I checked to see if the driver and firmware versions were up to date, and found that both needed to be upgraded. To install the driver version (1.20), I installed the latest Atto configuration tool (2.61) and flash file (FlashBundle_2004_03_31.24X), both of which can be found in the support/download section at www.attotech.com. Make sure you're running Mac OS X 10.2.4 or later (although 10.3.x is required to use the FC-24XL) and Java update 1.4.1 for the configuration tool to work properly.
I used the configuration tool to update the flash file on the FC-24XL and it worked without incident. Extracting and installing the new device driver required a few mouse clicks. A reboot was required due to the changes in the flash file.
Ordinarily, Mac OS X device drivers are loaded and unloaded dynamically or manually using the kextload and kextunload commands. After rebooting and evoking the configuration tool, I could see that the device driver and flash file had been updated.
|Atto Technology Celerity FC-24XL|
The Celerity FC-24XL is a high-performance host bus adapter suitable for applications that use large files and require high throughput.
Usability and interface
To better understand Atto's Celerity FC-24XL, we'll map the functionality of the HBA to the five layers (FC-4 to FC-0) of the FC networking model to see how it stacks up against the FC standards. (See "Fibre Channel networking model")
FC-4. Device drivers operate at the FC-4 layer by packaging information units (e.g., SCSI command/data/status) to be transported down to the other layers and out over the glass wire. Because device driver code is often accessed in main memory, it's important that this code is small and efficient. The Mac OS X device driver for the Celerity FC-24XL is just over 200KB, due mainly to the functionality of Mac OS X's I/O developers' kit.
In contrast, the packaged driver for the Windows platform is more than 4,000KB. FC-3. The still-developing FC-3 layer is described as a set of common services that serve multiple ports on the same node. Encryption, the striping of outgoing data across multiple ports, and the definition of multicast and hunt groups can all be done at this layer.
The potential to exploit the functionality of the FC-3 layer with the Celerity FC-24XL is great. For example, a real-time application that needs the greatest bandwidth with the least latency could be configured to access a four-disk LUN through each of the HBA's four ports that are connected to four separate fabric ports that are ultimately connected to four separate storage ports. This is disk striping at its best because the probability of overrunning a fabric or storage port drops to almost zero with the proper buffering at each interconnect.
In addition, using the features of the Alias Server (a well-known service in the fabric), frames destined for the node can now be directed to all four ports (multicast) or the least busy port (hunt group). A storage virtualization product can take the Celerity FC-24XL's four ports and present them as one logical path from a storage server to its client application hosts in the form of a hunt group. When application hosts access their storage over this path, load balancing is automatically achieved by the native functionality of the hunt group.
FC-2. The framing and signaling portion of the standard, FC-2 receives information units from the upper layer protocols and packages them into frames to be sent to the destination port. Depending on the class of service in use, some level of signaling is used to either ensure that the frame arrived, or that there are sufficient buffers for the delivery of the next frame.
The Celerity FC-24XL only supports the Class 3 connectionless service, and thus doesn't guarantee the delivery of frames with receipt acknowledgment. Therefore, upper layer protocols that don't manage delivery errors (e.g., IP/UDP) aren't suitable applications.
FC-1. FC-2 frames are eventually serialized at the FC-1 layer and beamed across the physical interface for delivery to the destination port. Serialization is performed immediately before transmission, and is used to establish a clock frequency between the communicating ports by encoding timing bits into the data stream. It's also used to maintain DC balance (equal number of zeros and ones) on the glass wire. The quality (jitter) of the rise and fall times of the optical signal is a direct reflection of the quality of the serialization-deserialization (SERDES) function on the HBA.
When considering SERDES functionality, focus on the number of integrated circuits per independent channel, power dissipation and the number and speeds of protocols supported by the circuit. This helps define an upgrade path for the HBA, or even uncover the full range of protocols that can be transmitted using the HBA.
The FC-24XL is outfitted with two dual-channel FC controllers to perform serialization functions on behalf of the upper layer protocols. There's one SERDES channel for each port to support the independence of the data stream as it flows to and from its destination.
FC-0. This layer describes the supported variants of the physical interface and their associated speeds. Twisted pair, copper and fiber optic cabling are the supported media over which FC frames can be transported. The different connectors associated with the cabling options are also defined.
The FC-24XL uses four 2Gb/sec, small form-factor pluggable LC-type connectors to achieve maximum port density. Designed for high-throughput local applications, the HBA is outfitted with multimode lasers. Thus, multimode fiber optic cabling--preferably the 50 micron type--must be used to connect the Celerity FC-24XL to the fabric or end device.
|Fibre Channel networking model|
This diagram illustrates the five layers of the Fibre Channel networking model as defined by the industry standard. The high-performance design of the Celerity FC-24XL adheres to the model standards.
Built for performance
The Celerity FC-24XL boosts the performance of these layers with its Intel XScale processor. The design approach is similar to the TCP offload engines available on iSCSI storage adapters. Although much of the FC protocol is implemented in hardware and is immune to the overhead associated with processing IP headers, the additional features (port-to-port streaming and hardware RAID) offered by the FC-24XL benefit by having an integrated processor.
The Celerity FC-24XL also comes standard with 128MB of SDRAM (upgradeable to 512MB) for direct memory transfers from the storage device without involving the application server's CPUs. This may appear to be overkill, but for applications that are sensitive to CPU interrupts, this isn't excessive.
When plugged into a PCI-X bus slot operating at 64 bits and 133MHz, Atto says the FC-24XL is capable of sustaining 1GB/sec transfer rates between its attached devices and the CPU. Overprovisioning the bus with four 2Gb/sec (4Gb/sec full-duplex) ports is similar to how you would provision initiators over ISLs. Considering the likelihood of concurrent full-duplex communication across all four ports, overprovisioning this resource makes sense.
The Celerity FC-24XL has eight 512-byte buffer credits for delivery into the fabric. This may not seem like much, but because the HBA operates over the Class 3 connectionless service and the maximum payload of a FC frame is 2048 bytes, the initiator needs only to receive an R_RDY frame from the communicating device to send the next data frame. In addition, the HBA is outfitted with multimode lasers. So the laser's light propagation distance is limited to 300 meters, and the R_RDY frame received by the HBA after the first frame is sent will arrive before the HBA is done sending the second frame.
Both initiator and target mode is supported on the FC-24XL. When configured for initiator mode, the HBA sends frames into the fabric. When configured for target mode, the HBA receives frames from the fabric and could, for example, support a storage virtualization server.
The FC-24XL also supports SNIA's HBA API, a common API developed by the standards organization. This is intended to improve the manageability of HBAs in a SAN without regard to proprietary-vendor APIs and is an essential part of the SMI-S. Product offerings from vendors complying with SMI-S can be managed using other vendors' software, thus avoiding vendor lock-in. Conformity to this specification should be considered mandatory when considering SAN hardware components.
The Celerity FC-24XL comes with two transceivers; two additional transceivers may be purchased separately. The retail price is approximately $2,600, which puts it out of reach for application servers that have simple access and throughput requirements. It's appropriate for data-hungry applications in film production, high-end engineering applications or even general-purpose servers running storage virtualization target software.
Although priced moderately higher than other four-port FC HBA offerings, such as LSI Logic Corp.'s LSI7402XP-LC and QLogic Corp.'s QLA2344, the Celerity FC-24XL sports upwards of 512MB of onboard SDRAM memory for media-rich applications that can take advantage of direct memory access.
The FC-24XL is a high-end product designed for the most data-driven applications. The density and speed of its ports provides unbridled access to storage content, especially large media-rich files. In addition, storage virtualization and disk-to-disk backups are also potential applications that can make use of the number and speed of the ports. Its adherence to FC standards at every level of the reference model should help allay compatibility concerns.
The performance characteristics of the four ports when combined with the Intel processor, multiple dual-channel SERDES circuits and up to 512MB of DMA memory makes the Celerity FC-24XL a must-have for applications plowing through terabytes of data daily, such as audio-visual content servers. You'll pay a premium for the Celerity, but if you're running demanding applications, you're likely to find that it's worth the price.