Storage.com

Fibre Channel over Ethernet (FCoE)

By Paul Kirvan

What is Fibre Channel over Ethernet?

Fibre Channel over Ethernet (FcoE) is a storage protocol that enables Fibre Channel (FC) communications to run directly over Ethernet. FCoE makes it possible to move Fibre Channel traffic across an existing high-speed Ethernet infrastructure and converge storage and IP protocols onto a single cable transport and interface. The goal of FCoE is to consolidate input/output and reduce switch complexity, as well as to cut back on cable and interface card counts.

Traditionally, organizations have used Ethernet for Transmission Control Protocol/Internet Protocol, or TCP/IP, networks and FC for storage area networks (SANs). Fibre Channel protocol supports high-speed data connections among computing devices that interconnect servers with shared storage resources and between storage controllers and drives via a SAN. FCoE shares Fibre Channel and Ethernet traffic on the same physical cable or lets organizations separate Fibre Channel and Ethernet traffic on the same hardware.

Fibre Channel over Ethernet uses a lossless Ethernet Fabric and its own FCoE frame format. It retains Fibre Channel's device communications but substitutes high-speed Ethernet links for Fibre Channel links between devices.

FCoE works with standard Ethernet cards, cables and switches to handle Fibre Channel traffic at the data link layer. It uses Ethernet frames to encapsulate, route and transport Fibre Channel frames across an Ethernet network via Ethernet ports from one switch with Fibre Channel ports and attached devices to another, similarly equipped switch.

FCoE is often compared to Internet Small Computer System Interface (iSCSI), an IP-based storage networking standard.

How Fibre Channel over Ethernet works

FCoE sends Fibre Channel packets across an Ethernet network. It accomplishes this by encapsulating the native Fibre Channel packets inside Ethernet packets

Figure 1 is an example of SANs that are connected to both FC and FCoE networks. Legacy storage devices remain connected via FC, and newer devices run on FCoE. The SAN interface in the diagram supports both FC and FCoE.

To make this work, a special type of network adapter called a converged network adapter (CNA) is used. A CNA combines the functionality of a Fibre Channel host bus adapter (HBA) with that of an Ethernet network adapter. The CNA is represented by the SAN interface in Figure 1.

The CNA not only provides the required physical connectivity, but also enables lossless Ethernet. This is essential because Fibre Channel is a lossless protocol and SANs expect lossless communications.

Who uses FCoE?

Fibre Channel typically appears in data centers where block storage devices are connected via fiber optic cables, owing largely to their bandwidth. Fiber optic cables have a distance limitation which can be addressed when Ethernet is used.

Assuming the SAN and associated storage devices are situated within a data center, FC can be used effectively. If storage devices are located outside a data center -- for example, at another campus location or elsewhere in the building -- FCoE is a good alternative because it can handle greater distances without any degradation in bandwidth.

Organizations that need to connect storage devices but wish to keep costs down might opt for FCoE, because the cost for a twisted pair cable used with Ethernet is lower than that of a fiber optic cable, not counting costs for FC switches and connecting devices. It's also easier for twisted-pair cable to go around corners, whereas fiber optic cable must observe specific radius limits when traversing corners or other cable routes.

Benefits of Fibre Channel over Ethernet

There are a number of benefits to using FCoE, including the following:

• Reduced costs and complexity. Because storage-related traffic is being sent over a conventional Ethernet network, FCoE eliminates the need for Fibre Channel switches, which reduces both the architectural costs and some aspects of network complexity.
• Increased performance. Fibre Channel is a high-speed storage protocol, with some storage devices supporting speeds of up to 128 gigabits per second (Gbps). Even so, such devices tend to be expensive. While some organizations might still use 8 Gbps storage devices and SANs, FCoE makes it possible to achieve speeds up to 100 Gbps by using comparatively inexpensive Ethernet networks.
• Simplified network management. Many tools exist for managing and monitoring Ethernet networks. Because FCoE routes storage traffic across a standard Ethernet network, an organization can use its existing networking tools to manage storage data traffic.

Challenges of Fibre Channel over Ethernet

There are several challenges that organizations might encounter when using FCoE, including these three:

• Network latency. FCoE relies on Ethernet for data transport, which introduces additional network latency compared to traditional FC connectivity. Ethernet networks can have inherent latency due to factors such as packet processing, congestion and network equipment. Latency can cause performance and responsiveness issues.
• Possible complexity from convergence. FCoE enables the convergence of storage and data networking over a shared Ethernet infrastructure. However, this convergence can introduce complexity in terms of network design, configuration and troubleshooting.
• Compatibility and interoperability. FCoE requires compatibility and interoperability between different vendors' products, including Ethernet switches, FC switches, HBAs and storage arrays. Ensuring seamless integration and compatibility among various components can be a challenge, particularly when dealing with older or heterogeneous infrastructure.

FCoE use cases

The following are examples of how FCoE can provide value to a storage infrastructure.

• Driving down costs. The main reason for using FCoE is to lower costs. FCoE eliminates the need for costly Fibre Channel switches, reduces power and cooling costs and simplifies storage managment.
• Ease of deployment. As noted, Ethernet cables can easily negotiate tight bends and difficult cable routes, whereas fiber optic cables are more limited and difficult to manipulate.
• Legacy hardware. Another use case for FCoE involves using legacy FC hardware. An organization might be able to use FCoE to improve performance without having to invest in modern Fibre Channel hardware.

FCoE vs. other storage protocols

FCoE has similarities to the iSCSI protocol. However, while FCoE encapsulates native Fibre Channel packets inside Ethernet packets, iSCSI encapsulates native SCSI commands inside IP packets.

Another difference between these two storage network protocols is that FCoE is designed to allow Fibre Channel communications across a high-speed Ethernet network and is generally used within the confines of a data center

On the other hand, iSCSI can be used for communications among devices within a data center, and can also be used as a means of connecting to a remote storage device. Additionally, FCoE requires a converged network adapter, while iSCSI can be used with any standard network interface card.

The future of storage is in collaboration and unified management capabilities. Find out what to look for with your next storage investment.