This article can also be found in the Premium Editorial Download "Storage magazine: Data archiving in the cloud."
Download it now to read this article plus other related content.
Supporting storage architectures
We’re seeing an evolution in storage architectures to help deal with the increasing volume of data associated with big data. Each has slightly different, but overlapping, characteristics.
On the I/O-intensive, high-transaction volume end, ESG sees a broad adoption of architectures that can scale up by adding spindles. That’s the traditional approach and systems like EMC VMAX, Hitachi Data Systems VSP and IBM DS8000 do well here.
On the large data size front, bleeding-edge industries that have been dealing with big data for years were early adopters of scale-out storage systems designed with enough bandwidth to handle large file sizes. We’re talking about systems from DataDirect Networks, Hewlett-Packard Ibrix, Isilon (now EMC Isilon) and Panasas, to name a few. Traditionally, scale-up implied there were eventual limits; scale-out has far less stringent limits and much more flexibility to add capacity or processing power. As big data sizes become more of a mainstream problem, some of these systems are finding more mainstream adoption. These more mainstream environments can be a mix of I/O- and throughput-intensive performance demands, so both scale-up and scale-out are often needed to keep up.
Finally, on the content volume front, we’re seeing more adoption of scale-out, object-based storage archive systems to make it easier to scale to billions of data objects within a single, easily managed system.
This was first published in May 2012