From DAS, to clustered databases, to virtualized storage from 3Par Data Inc. and a large clustered NAS system from Isilon Systems Inc., the process MySpace went through in just a few years could be the future of the entire industry as storage grows across the board, Whitcomb said.
In the very beginning, Whitcomb said a simple cluster of two servers to balance loads on the sites' Oracle databases was adequate. "And if you can get away with it," he said, "That's the architecture I still recommend because of its simplicity."
The site stuck with that model even as it added an EMC Corp. Clariion SAN to replace its DAS.
"Horizontal partitioning, we found, is very scalable," Whitcomb said. "In 2004, IBM did some tests showing that databases were capable of up to 3.2 million transactions a minute, but our database cluster this past March was doing 4.5 million transactions a minute."
But as the site grew to 10 million users, MySpace hit another ceiling.
"That's our theme, at MySpace," Whitcomb said. "We max everything out."
Even partitioning its SAN demanded too much time and "hot spots" were cropping up on frequently accessed disks. Eventually, MySpace let go of the traditional EMC storage to go with 3Par's Inserv SAN box, which includes a virtualization layer that automatically balances data across physical disks -- no more hot spots, no more time spent partitioning.
But servers and file sharing was still a problem. The clustered and continually repartitioned databases were becoming too cumbersome to manage; shared file systems also grew too big to take care of manually.
"I had engineers up all night walking a single file tree from one piece of hardware to another," Whitcomb said. "We were stuck."
It was at last year's SNW, Whitcomb said, that he saw a presentation by the Kodak EasyShare gallery -- a business very similar to MySpace, he noted -- about its use of clustered NAS systems from Isilon , and a light bulb went on.
"When I was rehearsing today's talk at home," he told the SNW audience, "My girlfriend remembered Isilon because I had come home last year so excited about them."
According to Whitcomb, Isilon had done for them with file sharing what 3Par had done for disk partitioning, balancing files across multiple nodes and allowing new nodes to be added with a minimum of management.
"We consider this a long-term solution because we can have up to 500 terabytes in a single file system and just keep adding more nodes," Whitcomb said. "And we have less than one full-time engineer managing it."
MySpace now serves its MP3 and video streaming features through Isilon's clustered NAS at a rate of 600 Mbps per node, giving an aggregate throughput of 10 Gbps for the whole system.
They'll need it -- MySpace is projected to boast more than 100 million users by January 2007; 250,000 new users join every day. MySpace is now looking at internationalization and a globally distributed network.
"You'll be seeing MySpace China soon," Whitcomb said.
Earlier at SNW, IDC analyst Doug Chandler predicted that more MySpace-style businesses would begin to grow, as well, and more and more storage vendors would find themselves in uncharted territory as they sell their storage into "instant coffee" Web-based storage resources like MySpace and various photo-hosting sites.
"This is a long-term trend for storage OEMs," Chandler said. "They are going to have to understand that they will become the arms suppliers -- their partners' partners will be selling their capacity, and it won't necessarily be visible to Wall Street."