Unilever Global, one of the world's largest consumer products companies, is making Virtual Instruments' NetWisdom performance monitoring tool a permanent part of its Fibre Channel storage-area network (FC SAN) infrastructure after it helped the IT staff resolve storage performance issues last year.
Headquartered in London and Rotterdam, The Netherlands, Unilever Global is the parent company of several household name brands, including Bertolli pasta, Lipton Tea, Slim-Fast diet drinks and Dove soap. Its data centers are located in northern England (company officials declined to specify the exact location for security reasons) and host approximately 5 petabytes (PBs) of data on a large and complex, multilayered FC SAN fabric.
Unix administrator Paul Faid said the company last year experienced storage performance slowdowns in its SAN infrastructure that were ultimately traced to overloaded ports on some of its Brocade Communications Systems Inc. switches, purchased through Hewlett-Packard (HP) Co.
At first, it was difficult to isolate the problem given the size and complexity of the fabric. "One server one day may experience throughput issues, and might not the next day," Faid said. "Different servers were having problems at different times."
It was also not an option to swap out whole switches. "This is a high-density environment always configured for HA [high availability], and downtime must be scheduled," said John Belooze, Unilever's enterprise hosting manager.
The problem was ultimately isolated to particular ports, and the IT team rebalanced paths in the fabric to eliminate the bottleneck without having to swap out switches or take the fabric down. "It was a lesson learned for us when our vendor, HP, brought a certain tool in to use themselves in evaluating the network," Faid said.
That tool was NetWisdom, which taps into the Fibre Channel layer to diagnose and troubleshoot the SAN infrastructure. Unilever began testing the product itself shortly after that, and installed it in production in June.
Virtual Instruments is a spinoff of Finisar Corp. which makes fiber-optic communication and test and measurement equipment for high-speed networks. Virtual Instruments adds the NetWisdom software that can either stand alone to report on storage performance, or use Finisar's Traffic Analysis Point (TAP) network sniffer that sits between a switch and SAN, copies all SAN traffic, strips away the payload, and dumps header information into a database for analysis and diagnostics.
Unilever first installed TAPs at the connection points between the IBM SAN Volume Controller network storage virtualization device it uses to pool storage in its IBM DS8000 series, and HP XP and EVA disk arrays. These TAPs should help the team identify storage performance problems before they impact users.
The TAPs has also helped Unilever smooth tuning and optimization of new equipment in the environment, Faid said. Unilever recently purchased a new virtual tape library (VTL), HP's Sepaton-based StorageWorks Virtual Library System (VLS), and NetWisdom helped get it configured quickly.
"You have to make sure you optimize the channels to balance I/O to the disk spindles on the back end," Faid said. "It usually takes a long time, sometimes days, of trying [different configurations] and testing them to ensure the best effective throughput." With a TAP on the link between the network and the VLS, the team had the new VLS up and running in a matter of hours.
Unilever is also using NetWisdom to evaluate new products – such as IBM's XIV Storage System array, as well as "fact check" vendor-supplied reporting software like HP's Storage Essentials or IBM's Tivoli Storage Productivity Center (TPC). So far, no discrepancies have been found.
Considering the nature of the product, it's no surprise that NetWisdom takes careful planning when setting up.
"There are some design ports you need to consider," before deploying a TAP, Faid said. "Putting a TAP on every link can be disruptive and expensive." The fabric is designed for high availability, including dual fabric links and multipathing software, but "interrupting a switch [to install a TAP] takes particular care – it's always fraught when you pull cables out."
To lessen the risk, Faid said, Unilever plans to TAP new SAN links it wants to monitor at deployment rather than when they're already up and running.