Generating a billion files to prove a point is no trivial task.
Nasuni Corp. claimed to have spent 15 months creating and testing a single storage volume within its service with more than a billion files, including 27,000 snapshot versions of the file system. Nasuni Filers captured the data and did the necessary protocol transformation to send it to Amazon’s Simple Storage Service (S3) and store the files as objects.
The Natick, Massachusetts-based company used a test tool to generate the files directly on the Nasuni Filers, starting with a single entry-level system and eventually ramping up to a total of four filers to ingest the data faster. The two hardware and two virtual filers were located in Natick and nearby Marlborough, Massachusetts.
Nasuni CEO Andres Rodriguez said the files were representative of customer data, based on information from the company’s tracking statistics. He said there has been pressure on his company and competitors to demonstrate the scale of file systems, as customers increasingly want to deploy a single file system across multiple locations.
“We’re going after organizations that are running, say, Windows file servers in 50 locations and each of those Windows file servers may have 20 or 30 million files,” Rodriguez said. “They’re having problems with their backups or with their Windows file servers running out of room or running out of resources.”
Rodriguez said the UniFS global file system within Nasuni Filers at each site gives them access to their millions or billions of objects stored in Amazon S3 or Microsoft Azure. He said it doesn’t matter if the Nasuni Filer is a “tiny little box” or a “tiny little machine version.” “No matter how little the Nasuni Filer is, it can still see, access, read, write the one billion files,” said Rodriguez.
How big a deal is the billion-file proof point?
Marc Staimer, president of Dragon Slayer Consulting in Beaverton, Oregon, viewed the Nasuni test as simply a “nice marketing assertion.”
“I commend them for running the test,” he said. “But, vendors such as EMC Isilon, Joyent, Panzura and other highly scalable scale-out file systems with global namespace can also provide access to all files from any node. A Nasuni filer is slower and primarily a gateway to objects stored in Amazon S3 or Microsoft Azure.”
Nasuni provided no performance information related to the billion-file demonstration. The company said only that data input/output performance varies based on the model of Nasuni Filer used. Higher end models support higher performance than entry level units, a company spokesman said.
Steve Duplessie, founder of Enterprise Strategy Group Inc. in Milford, Mass., said via an e-mail that Nasuni takes aim at second or tier-2 files, and performance is a “non-issue” with that class of data. He said Panzura is probably closest in approach to Nasuni but plays at a different level and has a heavy hardware footprint. He said Isilon can scale to a billion files but not globally. Isilon and Panzura cater to primary tier-1 data and carry the price tag to match, he said.
“If you were performance sensitive, you should use Isilon or NetApp,” said Duplessie. “Having said that, the overwhelming percentage of data in the organization is not performance sensitive, and the cloud is a fine place to keep it.”
Gene Ruth, a research director in enterprise storage at Gartner Inc., said he fields calls on a frequent basis from legal firms, construction companies, government agencies and other clients trying to provide common access to file environments from dozens, hundreds and in some cases thousands of branch offices.
“Nasuni is addressing the bulk of the market, which is support for universal access to files – being able to get at files on any device from anywhere. You have a common authoritative source that’s synchronized in the backend that provides those files,” said Ruth. “And they’re not the only ones that can do this.”
Ruth doesn’t view Nasuni’s billion-file announcement as significant, but he does see it as an indicator of the continuing evolution of what he calls cloud-integrated storage and what others often refer to as cloud gateways.
“Nasuni’s proven a point,” said Ruth, “that incrementally they’re getting bigger and more capable and more credible in addressing a bigger audience.”