Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Why object-oriented storage and interactive apps don't mix

Object-oriented storage has definite benefits, such as high scalability, metadata that allows users to easily find files and erasure coding for efficient data protection. But as Marc Staimer, founder of Dragon Slayer Consulting, discussed in this Tech Talk video, certain drawbacks remain.

"You don't want a lot of interactive applications" on object storage devices, Staimer noted. In general, he said, object storage has lower performance than file storage systems. This is partly due to the metadata and erasure coding found on object storage systems, which can increase latency for read and write operations. This latency means high-intensity applications will generally be better served by faster storage systems, making object-oriented storage better suited for storing large amounts of data.

"Applications that write to NFS or SMB natively tend to [perform a little bit better] than what most object stores can handle," he noted. So if your organization is heavily reliant on a Network File System or Server Message Block protocol, object-oriented storage may not be the best fit.

Another issue with mixing object storage with high-intensity applications is that there are often problems with eventual consistency in object technology. When you write data to an object-oriented storage system, all the nodes may not update at once. If you attempt to "read the data on one node that hasn't been updated, while it's being written to a different node," Staimer explained, "you can be reading old data." He argued that the time gap between nodes updating isn't particularly large, but it's still a significant enough interval to harm application performance.

Editor's note:

This video is Part 2 of a two-part video series on object technology. Part 1 provides an overview of some of the most effective object-oriented storage use cases buyers should be familiar with.

View All Videos

Transcript - Why object-oriented storage and interactive apps don't mix

Hi, my name is Erin Sullivan. I'm the assistant site editor for TechTarget's Storage Media Group. Joining me to discuss object storage is Marc Staimer, founder of Dragon Slayer Consulting. Thank you for joining us today, Marc.

Marc Staimer: My pleasure to be here, Erin.

Can object storage replace the traditional file system?

Staimer: Yes, it can. The traditional file system, it's got an interesting play in the market today. Generally speaking, the original shared storage with a file system came from servers like Microsoft Windows. It was your D: drive on your laptop or desktop. Great stuff. It enabled you to share files.

Today, most people prefer to share files with a sync and share. Why? Because it's easier. They can share [data] among different devices and different people. You can authorize who has access to specific data, not a file share. It's a little bit simpler in that aspect. So now you're looking at an application perspective, and many applications that write to NFS or SMB natively, tend to [perform a little bit better] than what most object stores can handle, to be candid with you.

But for those who don't require the performance, you can get an SMB, you can get an NFS interface in most object stores. In those that don't [have SMB or NFS], you can always get a gateway. But generally speaking, you can replace a traditional NAS system with an object store. [Object storage] happens to be more scalable, more durable and has a lower cost, but you can replace it.

What characteristics should you look for in an object storage system?

Staimer: It depends on your requirements. If your requirements are in the entertainment and media space, they're going to be different than they are in the life sciences. They're going to be different than they are in manufacturing. They're going to be different than they are in health services. They're going to be different in government. So it depends on your requirements.

Generally speaking, you're going to want flexibility. You want the ability to be able to use their hardware or your own hardware or somebody else's hardware. Generally speaking, you're going to want to have an [Amazon] S3 interface because many applications use an S3 interface. Generally, you're going to want erasure codes capable. In some cases, multi-copy mirroring. You maybe even want an NFS or SMB gateway, but those are general requirements.

Specifically, it depends on how well it's integrated. Do you want it integrated with Hadoop? Do you want it distributed with Hadoop? Do you want an HDFS [Hadoop Distributed File System] interface? Do you want it integrated with something like StorNext? Do you want it integrated with a specific application? That's going to change which object storage you're going to look at.

How does object storage scale differently from other storage systems?

Staimer: The way it scales differently is that it has a single name space that has no bottleneck. So because it's a shared-nothing architecture, you've got no index that is defining how you can scale it, how big you can scale it or how you can test it. There is no defined limit in object storage.

There are defined limits in file storage. There are defined limits in block storage on how big you can grow it adequately, whether you're talking scale up or scale out. With object storage, it's a scale-out, shared-nothing architecture, meaning there is no bottleneck ever between the scale and the performance and the capacity. In reality, every node you add, adds capacity and performance, positively.

How can eventual consistency become a problem in an object storage environment?

Staimer: Basically, when you write the data, and this is what object storage does in general, it will eventually upgrade or update all the nodes. So if you go to read the data on one node that hasn't been updated, while it's being written to a different node at that time, you can be reading old data. And that can create an error. Eventually, it'll be updated, but that can create an error, especially if you make a change to the data on the other node. Now you can get some interesting errors going on. It's not a huge time gap, but it is a time gap.

So you don't want a lot of interactive applications necessarily. You wouldn't want, let's say, a SQL database in transaction mode working with [object storage] because of that lack of immediate consistency. Now, there is one exception to that. There is one object storage vendor, Joyent with its Manta, that does provide immediate consistency. But generally, most object stores do not.

Can object storage be considered software-defined storage?

Staimer: Of course. Software-defined storage, unlike software-defined networking, is not a standard definition. Every vendor has a different definition and it pretty much fits their own stuff. Storage has always been defined by software, so the real definition of software-defined storage, lowest common denominator among all of them, is that the software that controls the storage hardware is separate from the hardware it controls.

Generally speaking, that's what all object storage is. They run on x86 server nodes, white-box commodity hardware, white-box commodity hard disk drives and flash drives. They don't care. Even though you may buy it packaged, generally speaking, it's pure software.

Thanks, Marc.

Staimer: You're welcome.

That wraps up today's Tech Talk on object-oriented storage with Marc Staimer. Thank you for joining us.

+ Show Transcript

Join the conversation


Send me notifications when other members comment.

Please create a username to comment.

What are the most practical uses of object-based storage?
Well, there is a lot to split hairs with Mr. Staimer on his comments about object-based storage. First, he gets away with claiming interactive applications and not suited for object storage without giving an definition of example.
Well, sorry for fat-fingering the keyboard with my previous shortened comment. Mr. Staimer claims that so-called interactive applications are not suited for object storage, but he doesn't define "interactive application" or provide an example. Is an enterprise file sync-and-share application interactive? To be clearer he could have said that transactional data is not suitable (today) for object storage. Metadata has little to do with latency. Erasure coding could have an impact on latency depending on the number of data fragments and parity fragments used to erasure code an object. Most object storage software can work with internal or external gateways that provide NFS and SMB support for legacy application access. With regard to data consistency, the CAP Theorem states you can only optimize for two out of three when it comes to data consistency, data availability and partition tolerance in a distributed storage service. Most object storage software vendors will choose data availability and partition tolerance over consistency. That said, if the object storage cluster is in single location, consistency should not be an issue if replication or erasure coding is used to store objects. If you are using object storage in a multiple data center environment, then using NFS or SMB gateways to handle file-locking while providing a local cache would be appropriate. Also, over time SSDs will replace HDDs in object storage or at least you will be able to implement an SSD tier for your hot and warm data and use an HDD tier for cold and archive data. The future of data storage is flash and object.