The report, AutoCAD 2007 vs. AutoCAD 2004: The effect on Wide-area Data Services Solutions, publishes the results of testing on Cisco's WAAS, Blue Coat's Mach 5, Silver Peak's NX series and Riverbed's Steelhead devices. The products were tested under a variety of conditions using WAN links of 1 Mbps and 10 Mbps, setting incremental save percentage (ISP) to zero and 50, saving major and minor changes, and running AutoCAD 2004 and AutoCAD 2007 versions.
The test results show that against a control of 8.1 MB of bandwidth for saving an 8 MB AutoCAD 2007 drawing file across the WAN with no WAN optimization device, Riverbed's bandwidth was at about 7.9 MB, while Cisco WAAS hovered around 8 MB, Silver Peak at about 8.2 MB and Blue Coat at close to 10 MB. "We were surprised to see some of our competitors were actually sending more data across the wire than the original file, let alone reducing it," said Alan Saldich, Riverbed vice president of marketing.
According to the Taneja/Riverbed test results, all WAN optimization products offered time savings to some extent on AutoCAD 2007 and 2008 drawing files. However, it found that all products that use data deduplication are affected by the new file formats.
Taneja Group analyst Jeff Boles said he participated in a live test of Riverbed's and Silver Peak's equipment at Riverbed's facilities in San Francisco. Riverbed is a client of Taneja Group and Silver Peak is not, but Boles said the firm insisted on being in the room for the test before it would validate Riverbed's results.
"This emphasizes the importance of understanding your entire business tack in the data center when managing technology," Boles said. "This kind of problem is going to keep popping up as layers of intelligence are added to the IT infrastructure."
Jeff Aaron, Silver Peak director of marketing, claimed in an email to SearchStorage.com that the numbers in Riverbed's report don't jibe with numbers Silver Peak has seen in the field, and questioned the Taneja Group's objectivity. "Make no mistake – this is a Riverbed report with jacked-up numbers," Aaron wrote. "This is by no means a valid third-party verification."
Competitors have criticized Riverbed and claimed their products don't have the same problems with the new AutoCAD files. Silver Peak released a report last month claiming superior results to Riverbed's in deduplicating the files. Packeteer (recently acquired by Blue Coat) also claimed superiority over Riverbed in AutoCAD environments, but Packeteer was never doing block or byte-level data deduplication, making the comparison apples and oranges, Saldich said.
More recently, Cisco provided a customer reference for its WAAS WAN optimization product to SearchStorage on the premise that the customer was not seeing the same problem as Riverbed customers had experienced. However, specific testing conducted with the help of Cisco service engineers revealed that the Cisco product suffered from the same inefficiencies as Riverbed's in the specific scenario described.
Riverbed argues that competitors have blurred the issue by exaggerating the problems with Steelhead appliances while understating their own difficulties. "We're talking about a very precise use case," Saldich said. "We're talking about AutoCAD 2007 or 2008 drawing files being saved over the WAN with an ISP of zero," meaning that the file's bytes are all scrambled when it's saved.