"There were no companies among their customer references who were our size," Honny said. "There were none in our vertical market, either."
The county had been using NAS boxes from Procom Technology (now owned by Sun Microsystems Inc.) for four years before a disk failure gobbled up 16 GB of critical city planning data, mainly property transactions that were later recovered after the Procom NAS hardware was repaired.
Honny said he first heard about Isilon while leafing through the pages of Storage magazine last year. (Full disclosure: Storage and SearchStorage.com are owned by the same company, TechTarget).
"At the time, they seemed the most mature in their product," Honny said. He began investigations in earnest, bringing Isilon's IQ 3000 product in for testing in early 2005.
"I think we were a new class of customer for them," he said. "We were more interested in the failover benefits of clustering our storage than the aggregate throughput. It's still a new concept even a year after we got ours."
So what convinced him to take the plunge?
"We approached it cautiously," Honny said. "We put in a very small implementation and put it through the most rigorous tests possible -- we unplugged nodes at random to test the failover capabilities. We tried to replicate the worst-case scenarios."
Honny said he put the company's president, Steve Goldman, through his paces in a phone call as well, learning the nitty-gritty details of Isilon's product roadmap and venture capital funding to that point.
In the year since, 15 terabytes (TB) of the company's 30 TBs of mission-critical file data, including images of birth, death and marriage certificates granted by the county, and every land and property transaction, and deed since 1980 have been migrated to the Isilon cluster. The other half of the data, still residing on the county's Procom systems, will be migrated in the next six months, Honny said.
The road hasn't been without its bumps. In the beginning, Isilon delivered a system with two Ethernet ports in each node but with the ability to make only one work. The dual ports are now working, but Honny said San Bernardino is waiting for Isilon to develop a way of sending only changes to files over the WAN to a secondary data center 15 miles from headquarters. Honny said for now, the company is using a third-party open source replication utility called RSync.
"Right now, Isilon sends whole files over the wire in big batches," Honny said. "You can throttle the throughput a little bit, but we'd like to see them build in some traffic controls as well."
A clustered future?
Isilon and its clustered storage competitors, Ibrix Inc., PolyServe Inc., Panasas Inc. and most recently, Crosswalk Inc., have been growing, thanks in part to a lag in a release by the major NAS vendor Network Appliance Inc. of its own clustered storage system known as OnTap GX. But analysts say the rising tide of a trend toward clustered NAS is also lifting all boats.
"Clustering isn't always about going bigger," said analyst Greg Schulz, founder of the StorageIO Group. "It's not appropriate for very, very small environments, but it offers flexibility and reliability that aren't limited in their appeal to the largest of environments, either."
Schulz said clustered storage systems could also appeal to the midrange user because they can be grown more incrementally than big boxes -- Isilon's smallest cluster configuration, for example, comes at 6 TB, and the smallest node at 3 TB. Users can also add performance separate from capacity with the IQ accelerator node.
"As data growth continues, clustered storage will grow more appealing in its ability to support higher capacity and throughput, as well as offer failover capabilities," said Brad O'Neill, senior analyst with the Taneja Group. "I believe that for a significant portion of enterprise NAS users, any new deployments in the next three to five years will be clustered systems."