File Systems: The state of the art


This article can also be found in the Premium Editorial Download "Storage magazine: Survey says storage salaries are climbing."

Download it now to read this article plus other related content.

Cluster vs. distributed/parallel
The key question faced by any user is: When should I care about a cluster file system vs. a distributed or parallel file system?

CFS are well designed for comparatively smaller node-count deployments in enterprise environments. Because of their integration with SAN architectures and their extreme focus on high availability and immediate recovery, a CFS is well suited to clustered databases and consolidation of file or application servers into a networked storage pool. Additionally, enterprise CFS have been increasingly coupled with NAS protocols (NFS, CIFS) to create scalable NAS environments or more accurately, "scalable NAS on SAN." Users should also consider CFS when high availability and recoverability for critical apps are absolutely essential.

DFS and PFS continue to dominate HPC environments. This is because they don't require SAN integration and are optimized for highly parallelized I/O to single clients. Additionally, this category of file-system technology is increasingly prevalent at the heart of scalable NAS offerings. These offerings typically leverage the DFS/PFS architecture to create an integrated plug-and-play cluster where every node behaves as an atomic member, contributing new CPU and storage to an overall system as they're added. Some DFS-based NAS offerings can achieve theoretical file-system sizes of more than 100TB and are demonstrating impressive real-world aggregate throughput.

Requires Free Membership to View

When client performance is critical or highly parallel I/O is a must, users should look to DFS/PFS solutions.

Both CFS and DFS/PFS will play important roles in the data center. CFS will undoubtedly continue to play a major role in enterprise virtualization initiatives because a CFS enables fine-grained scalability, and the sharing of multiple applications across all server and storage resources. Likewise, DFS/PFS will build traction as an engine for scalable NAS due to the easy atomic management capabilities these distributed file systems offer.

What's next?
Looking ahead 24 to 36 months, expect to see even more file-system innovations. Specifically, major vendors will integrate CFS technologies into their server and storage virtualization product families. Without easily deployed, transparent, shared file systems, all of the great utility computing dreams of major vendors will remain just dreams.

Accelerated by fast node-to-node interconnects, DFS/PFS will continue to boost performance across a range of I/O types, eventually challenging CFS for the lucrative clustered database market. This could have significant implications for the DBMS market as a whole when plug-and-play scalable databases become much easier to deploy.

Within three years, namespace aggregation technologies will become integrated features of servers, workstation file systems and within enterprise storage switching platforms. In short, unified namespaces will become ubiquitous, an essential part of an enterprise file-system deployment. Both CFS and DFS/PFS will also continue to extend their support for wide-area geographies. Data centers will begin to aggressively deploy both types of technologies to support branch offices for better collaboration and consolidation activities for file and block data.

The once lowly file system has finally blossomed into a state of increasingly diverse innovation. For storage professionals, the complexity has increased, but the good news is that file systems are finally starting to make their jobs easier.

This was first published in November 2005

There are Comments. Add yours.

TIP: Want to include a code block in your comment? Use <pre> or <code> tags around the desired text. Ex: <code>insert code</code>

REGISTER or login:

Forgot Password?
By submitting you agree to receive email from TechTarget and its partners. If you reside outside of the United States, you consent to having your personal data transferred to and processed in the United States. Privacy
Sort by: OldestNewest

Forgot Password?

No problem! Submit your e-mail address below. We'll send you an email containing your password.

Your password has been sent to: