Evaluate Weigh the pros and cons of technologies, products and projects you are considering.

Why object-oriented storage and interactive apps don't mix

Object-oriented storage has definite benefits, such as high scalability, metadata that allows users to easily find files and erasure coding for efficient data protection. But as Marc Staimer, founder of Dragon Slayer Consulting, discussed in this Tech Talk video, certain drawbacks remain.

"You don't want a lot of interactive applications" on object storage devices, Staimer noted. In general, he said, object storage has lower performance than file storage systems. This is partly due to the metadata and erasure coding found on object storage systems, which can increase latency for read and write operations. This latency means high-intensity applications will generally be better served by faster storage systems, making object-oriented storage better suited for storing large amounts of data.

"Applications that write to NFS or SMB natively tend to [perform a little bit better] than what most object stores can handle," he noted. So if your organization is heavily reliant on a Network File System or Server Message Block protocol, object-oriented storage may not be the best fit.

Another issue with mixing object storage with high-intensity applications is that there are often problems with eventual consistency in object technology. When you write data to an object-oriented storage system, all the nodes may not update at once. If you attempt to "read the data on one node that hasn't been updated, while it's being written to a different node," Staimer explained, "you can be reading old data." He argued that the time gap between nodes updating isn't particularly large, but it's still a significant enough interval to harm application performance.

Editor's note:

This video is Part 2 of a two-part video series on object technology. Part 1 provides an overview of some of the most effective object-oriented storage use cases buyers should be familiar with.

View All Videos

Transcript - Why object-oriented storage and interactive apps don't mix

Hi, my name is Erin Sullivan. I'm the assistant site editor for TechTarget's Storage Media Group. Joining me to discuss object storage is Marc Staimer, founder of Dragon Slayer Consulting. Thank you for joining us today, Marc.

Marc Staimer: My pleasure to be here, Erin.

Can object storage replace the traditional file system?

Staimer: Yes, it can. The traditional file system, it's got an interesting play in the market today. Generally speaking, the original shared storage with a file system came from servers like Microsoft Windows. It was your D: drive on your laptop or desktop. Great stuff. It enabled you to share files.

Today, most people prefer to share files with a sync and share. Why? Because it's easier. They can share [data] among different devices and different people. You can authorize who has access to specific data, not a file share. It's a little bit simpler in that aspect. So now you're looking at an application perspective, and many applications that write to NFS or SMB natively, tend to [perform a little bit better] than what most object stores can handle, to be candid with you.

But for those who don't require the performance, you can get an SMB, you can get an NFS interface in most object stores. In those that don't [have SMB or NFS], you can always get a gateway. But generally speaking, you can replace a traditional NAS system with an object store. [Object storage] happens to be more scalable, more durable and has a lower cost, but you can replace it.

What characteristics should you look for in an object storage system?

Staimer: It depends on your requirements. If your requirements are in the entertainment and media space, they're going to be different than they are in the life sciences. They're going to be different than they are in manufacturing. They're going to be different than they are in health services. They're going to be different in government. So it depends on your requirements.

Generally speaking, you're going to want flexibility. You want the ability to be able to use their hardware or your own hardware or somebody else's hardware. Generally speaking, you're going to want to have an [Amazon] S3 interface because many applications use an S3 interface. Generally, you're going to want erasure codes capable. In some cases, multi-copy mirroring. You maybe even want an NFS or SMB gateway, but those are general requirements.

Specifically, it depends on how well it's integrated. Do you want it integrated with Hadoop? Do you want it distributed with Hadoop? Do you want an HDFS [Hadoop Distributed File System] interface? Do you want it integrated with something like StorNext? Do you want it integrated with a specific application? That's going to change which object storage you're going to look at.

How does object storage scale differently from other storage systems?

Staimer: The way it scales differently is that it has a single name space that has no bottleneck. So because it's a shared-nothing architecture, you've got no index that is defining how you can scale it, how big you can scale it or how you can test it. There is no defined limit in object storage.

There are defined limits in file storage. There are defined limits in block storage on how big you can grow it adequately, whether you're talking scale up or scale out. With object storage, it's a scale-out, shared-nothing architecture, meaning there is no bottleneck ever between the scale and the performance and the capacity. In reality, every node you add, adds capacity and performance, positively.

How can eventual consistency become a problem in an object storage environment?

Staimer: Basically, when you write the data, and this is what object storage does in general, it will eventually upgrade or update all the nodes. So if you go to read the data on one node that hasn't been updated, while it's being written to a different node at that time, you can be reading old data. And that can create an error. Eventually, it'll be updated, but that can create an error, especially if you make a change to the data on the other node. Now you can get some interesting errors going on. It's not a huge time gap, but it is a time gap.

So you don't want a lot of interactive applications necessarily. You wouldn't want, let's say, a SQL database in transaction mode working with [object storage] because of that lack of immediate consistency. Now, there is one exception to that. There is one object storage vendor, Joyent with its Manta, that does provide immediate consistency. But generally, most object stores do not.

Can object storage be considered software-defined storage?

Staimer: Of course. Software-defined storage, unlike software-defined networking, is not a standard definition. Every vendor has a different definition and it pretty much fits their own stuff. Storage has always been defined by software, so the real definition of software-defined storage, lowest common denominator among all of them, is that the software that controls the storage hardware is separate from the hardware it controls.

Generally speaking, that's what all object storage is. They run on x86 server nodes, white-box commodity hardware, white-box commodity hard disk drives and flash drives. They don't care. Even though you may buy it packaged, generally speaking, it's pure software.

Thanks, Marc.

Staimer: You're welcome.

That wraps up today's Tech Talk on object-oriented storage with Marc Staimer. Thank you for joining us.

+ Show Transcript