The renovations project started in 2003 at the community college's Valley Glen, Calif.-based campus and included approximately $2.3 million to build new data centers. The college tore down the library where the old data center was located and built redundant high-available data centers – named Alpha and Omega – for load balancing about a year ago.
The data centers are almost identical, according to Yefrem Kozin, manager of college information systems. The only difference is that all wide-area network (WAN) connectivity (routers, firewalls, etc.) runs from the Omega high-available data center. Los Angeles Valley College uses two 10 GbE links between the data centers, and does three-way replication across the campus sites and to a remote disaster recovery (DR) site about 20 miles away in downtown Los Angeles.
A SAN upgrade was part of the move to the new data centers. Los Angeles Valley College had been a LeftHand Networks Inc. (Hewlett-Packard Co. acquired LeftHand last October) iSCSI SAN customer since 2003, and installed six new LeftHand P4000 nodes in each data center – three with SAS drives and three with SATA – for a total of 90 TB. Applications run on the SAS tier, while data is on the SATA drives.
Los Angeles Valley College also runs HP ProLiant blade servers and ProCurve 5412 and 8212 switches with 10 GbE modules. Kozin said that has improved support and helped the transition to the new data centers, although the SAN and local-area network (LAN) remain separate entities, each with its own set of switches, fiber optics and so on.
Kozin said the immediate benefit of 10 GbE storage is that the SAN is more efficient and easier to expand.
"Performance-wise, any snapshot or any addition we make to the SAN works smoothly because our network is 10 Gig," he said. "We have no delay in any application data. We keep all images on our storage for different departments, and we saw a big improvement when restoring images through the 10 Gig network."
Looking ahead, Los Angeles Valley College will store all video material on the SAN and its email capacity will expand as it gets into e-learning. Email capacity will continue to grow organically as well because the college allows students to keep their email accounts after they graduate. Kozin said it takes approximately 10 minutes to add a cluster to his LeftHand SAN.
Switch to 10 GbE a learning experience
Still, setting up the 10 GbE SAN was a learning experience for Los Angeles Valley College's IT staff.
"It was not an easy transition," Kozin said. "Before you go to 10 Gig, you should realize how 10 Gig works. We had a lot of dropped data, so we enabled flow control. By default, all HP switches come with flow control disabled, and it's important to make sure that flow control is enabled on the storage side. It's also important to know how 10 Gig modules work. On 5400 switches, modules can hold four 10 Gig transceivers. It's better to use port number one and number three for 10 Gig, and port number two for GigE, and not put the 10 Gig E ports together."
The transition was also delayed while Los Angeles Valley College waited for HP to release 10 GbE cards for its ProCurve switches. Kozin said he expected them last September, but they didn't arrive until December. In addition, those cards didn't work with the LeftHand modules the college had. LeftHand finally sent Kozin 12 upgraded devices for free in March.
Kozin said the switch to the new data centers in July 2008 went smoothly, and was carried out by the college's internal IT staff over two weekends without much downtime. He said the college was without services for one day over one of the weekends.
"While we installed all the new equipment – switches, enclosures and storage servers – we continued to run our network from the old data center," he said. "Then we joined new storage devices to existing clusters and moved data to the new storage servers. On one of the weekends, we moved the old storage servers into the new location and connected them to the new LAN and storage infrastructures. We used the HP [Insight] Server Migration Tool for physical and virtual servers to migrate some of the old servers into the new blade servers."
Along with 10 GbE, Kozin said thin provisioning also quickly allocates storage on his SAN.
"We allocate virtual server or a real server for every department, and we allocate space from a pool," he said. "If one department needs more capacity, they can buy their own storage, attach to our pool and we allocate the storage. Everything is centralized. With thin provisioning, data's not consuming any space until it's written. It saves a lot of work allocating storage because we just allocate more space without interruptions."