When TheStreet Inc. migrated 30 business applications and databases to Amazon Web Services to run a virtual private...
cloud, its IT team encountered turbulence because the clustered file system did not scale well enough to replicate millions of files.
The move to the cloud eventually worked out for the digital financial media company that distributes its content through online, social media, tablet and mobile channels, but the first four months were rocky. TheStreet first used the GlusterFS open source distributed file system to deploy Amazon Elastic Compute Cloud (EC2). That caused outages when instances fell out of sync across Amazon's availability zones during replication.
Last April, TheStreet switched from GlusterFS to SoftNAS, which provides network-attached storage (NAS) in the cloud. The company uses SoftNAS SnapReplicate for block-level replication instead of file-level replication.
"Amazon is a virtual cloud, so if you want applications to be fault-tolerant, then you have to architect applications to run across multiple availability zones. In the old way, we ran two data centers," said Stefan Chopin, The Street's chief technology officer. "[Amazon Web Services] is a fairly complex system. You upload once within your architecture and your replication depends on which system. An EC2 is different than a file store, which is different than [a] relational database service store."
SoftNAS is a ZFS-based scale-out file system configured on Amazon EC2, connecting Amazon Elastic Block Storage (EBS) instances to SoftNAS EC2 instances to serve up a file system via NFS, CiFS or iSCSI. SoftNAS software is designed to overcome performance and availability issues with EBS.
TheStreet started with three Amazon EC2 instances running on GlusterFS. Within weeks, it was down to one instance due to "split brain" issues, where the instances were out of sync and caused application servers to go down.
TheStreet's Chopin said the company switched to the Amazon cloud because it needed a more flexible architecture than the EMC SAN it used for more than 10 years. The Amazon cloud allows him to provision storage based on the Web server's needs. So if a server needs 10 GB file stores, he goes to the EBS console and provisions that amount of storage to that Web server. RAID levels can be designed for high performance for more demanding applications.
"But what you don't get is replication of content or synchronization to another availability zone," Chopin said. "So you need a file system that can do [that]."
TheStreet configures EC2 for RAID 10 to get better IOPS, throughput and redundancy out of EBS. A second SoftNAS instance was configured in a different availability zone as a failover target in case there is an outage in the primary availability zone. SnapReplicate only moves data blocks that have been changed from the primary SoftNAS controller node to the secondary node.
Scheduled file system snapshots also provide several weeks of recovery points. SoftNAS provides real-time backups for disaster recovery to secondary availability zones.
"Amazon is in the business of providing cloud computing," said Rick Braddy, SoftNAS founder and managing partner. "They don't provide a NAS or a fault-tolerant file system. We provide a primary NAS and we use block replication. We only replicate data changes at the block level instead of the file level. So, instead of sending all 1,000 files, we only send changes at the block level."