Hot Spots: Web 2.0 storage: Challenges and choices


This article can also be found in the Premium Editorial Download "Storage magazine: RAID turns 20: Do you still need it?."

Download it now to read this article plus other related content.

Do-it-yourself method (build or buy)
Google decided to build its own. Other companies, such as Facebook and MySpace, opted to buy products that supported their needs. Building out the infrastructure might make sense for Google because it has specialized expertise building massively scalable parallel storage systems and a culture that supports this type of approach. However, this isn't the preferred approach for most companies. Leveraging commodity disks and building out a proprietary file system takes years and lots of resources. If you believe your company has the talent, time, support and internal buy-in to take on this endeavor, you may want to consider this option. But ask yourself this question: Can you get your company's new service or product to market faster by building it yourself or with an off-the-shelf solution?

Again, the requirements for Web 2.0 models are very different from those associated with email or database applications. With Web 2.0, you're aiming to build a user population of hundreds of thousands (and potentially millions) of users. The data will be stored forever and will probably be filled with images, audio files and movies. If you choose to buy the infrastructure, where do you begin? Do you leverage your existing storage infrastructure that was really built for a different set of requirements or do you seek out next-generation storage systems built for

Requires Free Membership to View

these types of apps? These systems need to be massively scalable, provide predictable and reliable performance, and come at an affordable price point. With the potential to store billions of files and objects, they must also be easy to manage across multiple systems or geographies. In addition, they must interface with Web-based protocols and have replication capability.

One of the more interesting developments in outsourcing is related to Web 2.0. Outsourcing companies are providing not only the infrastructure and host facilities, but the services to help accelerate Web 2.0 initiatives. Companies like Amazon's Simple Storage Service (S3) and Nirvanix provide APIs to help customers handle multi-tenancy applications, file sharing, and video or image transcoding. While these services are relatively new (S3 has been around for approximately one year and Nirvanix just launched this fall), they're generating significant interest; they're basically Web 2.0 vendors that are enabling their customers to offer Web 2.0 services by using their infrastructure and processes.

This was first published in November 2007

There are Comments. Add yours.

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: