Dealing with big data: The storage implications


This article can also be found in the Premium Editorial Download "Storage magazine: Data archiving in the cloud."

Download it now to read this article plus other related content.

Supporting storage architectures

We’re seeing an evolution in storage architectures to help deal with the increasing volume of data associated with big data. Each has slightly different, but overlapping, characteristics.

On the I/O-intensive, high-transaction volume end, ESG sees a broad adoption of architectures that can scale up by adding spindles. That’s the traditional approach and systems like EMC VMAX, Hitachi Data Systems VSP and IBM DS8000 do well here.

On the large data size front, bleeding-edge industries that have been dealing with big data for years were early adopters of scale-out storage systems designed with enough bandwidth to handle large file sizes. We’re talking about systems from DataDirect Networks, Hewlett-Packard Ibrix, Isilon (now EMC Isilon) and Panasas, to name a few. Traditionally, scale-up implied there were eventual limits; scale-out has far less stringent limits and much more flexibility to add capacity or processing power. As big data sizes become more of a mainstream problem, some of these systems are finding more mainstream adoption. These more mainstream environments can be a mix of I/O- and throughput-intensive performance demands, so both scale-up and scale-out are often needed to keep up.

Finally, on the content volume front, we’re seeing more adoption of scale-out, object-based storage archive systems to make it easier to scale to billions of data objects within a single, easily managed system.

Requires Free Membership to View

The advantage of these systems is that they enable robust metadata for easier content management and tracking, and are designed to make use of dense, low-cost disk drives (Dell DX 6000 series is a good example here).

This was first published in May 2012

There are Comments. Add yours.

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: