Manage Learn to apply best practices and optimize your operations.

Visualizing data flow, part three: Managing storage over distance

Jim Booth reviews some options for moving data between geographic areas.

Jim Booth

Jim Booth is our storage administration expert and our newest storage management expert. Jim is director of systems engineering for Creekpath Systems.

If you have a question for Jim, enter it here (for storage administration) or here (for storage management).

Also, if you are looking for more on administration or storage management, view more of Jim's expert answers. Jim has also been contributing his input on storage management issues in our .EMQZaac7k06^1@.ee83ce3>Storage Management Tips & Tricks discussion forum.

Managing storage over distance

Visualizing data flow within the data center is a complex task. It becomes increasingly difficult when you introduce different connectivity options outside the data center. The purpose of this tip is to review some options of moving data between geographic areas. The primary technologies that will be discussed are Fibre Channel, SONET, and DWDM. We will not only look at how each of these technologies can be used to extend data connectivity, but how to visualize and manage these transports.

Part 1 of this tip discussed data flow and visualization. Part 2 described the use of metrics and removing the barriers to heterogeneous storage management through the evolving standards of CIM/WBEM. This part will focus on network technologies used to extend outside the data center and management considerations for each option.

A few options

Campus (Fibre Channel)

The most obvious channel extension method is to simply create a link between two SAN switches (ISL) at different locations. This technique works well in a campus environment since you can use hardware providers' tools to visualize and management the link.

This is a good campus solution with a few considerations:

    Distance: Additional requirements are long wavelength GBICs for distances greater than 1 Km and 9uM fiber that is more expensive than multimode.

    I/O Rate: Also realize that you are going to need single-mode fiber to obtain the 1063 Mbps throughput at a distance up to 10Km.

    Buffer: If there is a large enough buffer within the switches to compensate for in-flight data traffic, then you will not have a problem with this distance solution. But look for BBCreditZero faults.

Some management tools available today are from Brocade, McDATA, and third-parties that not only measure the I/O traffic, but also allow for planning and configuration management.

This practice is typically a do-it-yourself activity for a SAN engineer that has deployed a storage network consisting of one or more ISLs.


When using a leased-line from a Telco, things get a little more interesting. A SONET OC-3 connection can be a cost-effective transport over long distances and can be bought as either a fractional band or as a managed service. Most choose the later since there is no initial capital cost and service level agreements (with financial penalties) are happily agreed to in the current Telco arena.

You can deploy a distance solution like this yourself, but that will require hardware, management software, network connectivity, and transport expertise. The most commonly deployed hardware for this solution is the Nortel 3500 or Cisco 15540 system.

Management of the Nortel system is the Preside solution that allows visualization and management of the network infrastructure and applications by supporting end-to-end provisioning, alarm monitoring and correlation, and configuration control.

Cisco also has a management package is called the Cisco Transport Manager (CTM). CTM allows for optical administration, provisioning, configuration and performance monitoring. This management solution can scale to support up to 2500 Network Elements.

While both Cisco and Nortel provide excellent solutions, I still believe that a managed service for this distance solution is a far better value than investment in the hardware, software and personnel to keep a solution like this running.


Things get very interesting when we embark on a dark fiber solution (but interesting in a good way). Certainly a dark fiber solution offers the greatest flexibility and can be tailored to your exact needs. For data transport, DWDM is being exploited as a high-bandwidth, multi-service solution. If you are lucky enough to already have dark fiber between the two locations to be connected; utilizing a couple wavelengths or more can provide the throughput you require (OC-x per wavelength).

With DWDM you have a choice of either doing it yourself or purchasing a managed service. Unlike the SONET solution, I recommend working with a certified data transport integrator to build your DWDM solution. While a managed service could be less expensive, the lure of DWDM is the flexibility –-something that can be lost in the managed service.

While there are management tools for DWDM, they do not examine the wavelength itself, but the transport information at each end. Look for future management products to not only examine the data transport, but also storage networking.

Recommendations for management over distance

1. If you are going less than 10Km in a campus solution, fibre channel extension is probably your best option. Choose a hardware and software management platform that allows for detailed monitoring of port, buffer and fault information.

2. Managed services make sense when you do not want to invest in the hardware and personnel, but require a known service level for your storage over a given distance. Shop around for the best deal from a provider with management functionality that meets your needs.

3. DWDM, while more complicated, can provide the distance of managed services with the flexibility of local channel extension products. Management can be a challenge, but a well designed system from a knowledgeable integrator can tailor the perfect solution to your needs.

Dig Deeper on Data storage management

Start the conversation

Send me notifications when other members comment.

Please create a username to comment.