Big data storage

Find the latest news, tips and techniques for managing big data storage on this topics page. Big data storage management requires overcoming challenges, whether it’s carving out storage for applications such as Hadoop or high-capacity systems. Big data architectures include object storage and clustered NAS arrays that may need to store petabytes of data.

May 24, 2021 24 May'21
WANdisco introduces Hive metadata migration to Databricks

WANdisco LiveData Migrator can now move Hive metadata alongside Hadoop data to Databricks, saving customers from a lengthy and manual transformation process.
May 07, 2021 07 May'21
Belgian telecom picks Datadobi for NAS migration

Large-scale storage migrations can be difficult without the help of a software tool, as telecom Proximus Group found.
February 24, 2021 24 Feb'21
Next Pathway update targets Hadoop migrations

Next Pathway's data scanning and migration tools can now analyze Hadoop ecosystems and translate its codes to cloud-based data warehouses such as Snowflake and Amazon RedShift.
January 25, 2021 25 Jan'21
Vast Data storage packages Nvidia GPU for AI processing

Vast Data storage reference architecture combines the vendor's Universal Storage all-flash system with Nvidia DGX supercomputers. NFS multipathing over RDMA boosts AI data training.

Bring yourself up to speed with our introductory content

Hadoop as a service (HaaS)

Hadoop as a service (HaaS), also known as Hadoop in the cloud, is a big data analytics framework that stores and analyzes data in the cloud using Hadoop. Continue Reading
exbibyte (EiB)

An exbibyte (EiB) is a unit used to measure data capacity. Continue Reading
zebibyte (ZiB)

A zebibyte (ZiB) is a unit used to measure computing and storage capacity. Continue Reading

Weigh the pros and cons of technologies, products and projects you are considering.

An overview of Microsoft Project Silica and its archive use

Microsoft's Project Silica stores data in silica glass, similar to the crystals in 'Superman' films. In fact, 'Superman' already played a major role in the forward-looking project. Continue Reading
Big data vs. artificial intelligence: How they compare

Artificial intelligence can help synthesize and analyze the large volumes of information provided by big data initiatives. The two are different, but they work well together. Continue Reading
Converged secondary storage changes (almost) everything

Converged secondary storage brings important benefits but also crucial choices. Learn the advantages and what you must do, and what questions to ask, in order to reap them. Continue Reading

Learn to apply best practices and optimize your operations.

Using on-premises object storage beyond secondary storage

On-premises object-based storage is replacing traditional storage in areas such as streaming, big data and IoT. Get the latest on how this technology is being used. Continue Reading
Automated data management is a crucial part of IT's future

Automating data management will play an important role in helping us cope with the coming zettabyte apocalypse, a time when the amount of data generated will overwhelm storage resources. Continue Reading
Data governance best practices exploit big data

Emerging technologies, such as AI and IoT, in the age of big data mean there's more data than ever to benefit from and for data governance policies to manage and secure. Continue Reading

We’ve gathered up expert advice and tips from professionals like you so that the answers you need are always available.

The future of storage administration: 4 key questions answered

What are the best career moves a storage professional can make? Learn about the skills, technologies and strategies to focus on now for the most payback in the future. Continue Reading
High-performance object storage challenges in the modern data center

Object storage works well for long-term backup and archiving. See how high-performance capabilities are expanding it to high-scale, high-capacity workloads. Continue Reading
Hadoop data analysis: Common concerns with the HDFS platform

Using HDFS technology as a data analysis platform may be insufficient for your storage needs. Explore the challenges storage administrators may encounter and find out how to address them. Continue Reading