Sergey Nivens - Fotolia

Airfare analysis takes flight with PCIe flash storage

Airline Tariff Publishing Co. is building in-memory storage for airfare analysis using Dell PowerEdge servers and Micron P420m PCIe flash drives.

Seeking the fastest storage performance possible to help airlines build pricing models, Airline Tariff Publishing Co. (ATPCO) relies on PCIe flash storage to drive its analytics platform.

ATPCO uses Micron Technology Inc. P420m PCIe-attached solid-state drives (SSDs) inserted in Dell PowerEdge servers to drive its Fare Analytics Engine. The system helps airlines analyze their fares against those of competing carriers.

Micron flash gives fast storage in small footprint

Washington, D.C.-based ATPCO collates and presents airfare-related data from more than 450 domestic and international airlines. The fare information is then disseminated hourly to its subscribers, which include airline websites, content aggregators, computerized reservation services and global travel agencies.

ATPCO's system consists of 25 PowerEdge 630 storage servers, each one outfitted with a single Micron P420M flash card. The storage is built for speed and performance, rather than density, said Navid Abbassi, ATPCO's chief architect of information technology.

"The working storage for any given image is between 250 GB and 500 GB. That's what I need for each implementation of the fare analytics engine," Abbassi said.

The system is in beta testing internally at ATPCO and by a handful of beta customers in the airline and travel industries. ATPCO expects the system to go live in 2016.

When building the analytics capabilities, ATPCO first experimented with spinning media and running SSDs over Serial-Attached SCSI (SAS) links, but neither arrangement met performance requirements. Abbassi said he needed fast, robust storage in a small footprint to handle dynamic airfare data that changes between two million and 10 million times per day.

In-memory storage aids performance, RAID rebuilds

When an airline sets or modifies a fare, ATPCO receives the information and sends updates to its subscribers, such as Expedia, Priceline and other ticketing websites. All data changes are written to persistent storage for high availability and to enable fares to be analyzed quickly.

"We started out by looking at what each application needs to do," Abbassi said. "Our conclusion is that the applications should be manipulating all the data in memory to give the performance we need. I wanted a nonvolatile storage platform on the machine, so I can apply an update and load it into memory as fast as possible."

Using PCIe flash storage enables ATPCO to quickly process, sync and commit updates in memory.

ATPCO ran a series of comparison tests using PCIe flash storage from Micron and two undisclosed vendors. Micron was awarded the bid based on its product roadmap, controller technology and competitive pricing, Abbassi said.

Using PCIe flash storage enables ATPCO to quickly process, sync and commit updates in memory. Abbassi said flash in commodity servers also provides scalable storage with failover, eliminating the need to buy separate backup storage.

"Our failover architecture is to have at least two, maybe three or four machines for N+1 redundancy to handle the load in event of a single failure," he said. "When a failed server comes back with all the flash intact, then I don't need to recover anything except missing hourly updates. With Micron's flash drives, we can recover those updates on the fly to Micron flash in about two minutes, sometimes even less."

Next Steps

Memory channel vs. PCIe flash storage

PCIe flash storage pros and cons

PCIe flash cache management

Dig Deeper on All-flash arrays