TCO analysis: does it work for storage?

TCO analysis is all the rage, but use with caution—the numbers can be twisted by vendors trying to make a sale.

What does a storage solution actually cost? Ask a vendor and they'll most likely answer your question with another...

question: How much do you want it to cost?

It's a well-known fact that vendors, consultants, analysts and even CIOs themselves can manipulate the TCO analysis to make costly solutions look cheap, cheap solutions look costly and anything in-between depending on their particular agenda.

As a result, Senior Analyst Tony Prigmore of the Enterprise Storage Group in Milford, MA, says, "TCO is bad. We assess models and by and large they are invalid. We haven't seen a consistent level of credibility in any vendor TCO model." His recommendation to clients: "Do not bother with a TCO analysis unless you control the assumptions and data behind the model."

Manipulating TCO models

TCO models offer ample places to build in bias.

Acquisition price - the lower the price, the lower the TCO, but is this the price you actually get?

System sizing - vendors can undersize the system to lower the acquisition price and resulting TCO.

Estimated utilization - the higher the utilization the lower the TCO, but who is doing the estimating and how realistic is it?

Cost of management - the more people it takes to administer the storage the higher the TCO. This is a judgment call only you can make.

High availability - the greater the uptime, the lower the TCO. Is it 99.999 or 99.9? Be realistic.

Frequency of configuration changes - the more configuration changes, the higher the TCO. Again, this is a call only you can make, not the vendor.

Frequency of upgrades - even if the upgrade is included in the maintenance agreement, implementing the upgrade increases TCO.

Training - the more training required, the higher the TCO.

Inflation - this is a wildcard that can dramatically affect wages and other costs over the TCO timeframe, usually three years. Don't let the vendor low-ball the inflation factor.

Integration costs - it is easy to underestimate the time and effort involved.

Gartner Group, the Stamford, CT, consultancy, hopes to bring some needed credibility to the subject of storage TCO with a new storage TCO model. Seven vendors, ranging from industry leaders like Sun and IBM to start-ups like 3PARdata paid "six figures," according to Dan Hoffmann, the director of enterprise storage management at BMC Software Inc. to join with Gartner in the initial development of the model. The Gartner model extends the concept of TCO into ROI and is intended to become a tool for managing an enterprise storage environment. CIOs, however, still may have trouble definitively answering the basic question-what this storage solution will cost. The Gartner model allows vendors to modify data and parameters, which despite protections built into the model raises the possibility of introducing bias and muddying the results.

Fuzzy math

TCO can be an effective tool to identify and quantify all the costs associated with a technology purchase, but it hasn't been widely applied to storage. Beyond being susceptible to manipulation, TCO raises as many questions as it answers, and the numbers can get fuzzy. "The big question is whether the numbers are real," says Scott Robinson, chief technology officer, Datalink, Chanhassen, MN, an independent storage consulting firm.

"If the vendor supplies the TCO model, we take it with a very large grain of salt," says Rocco Esposito, chief technology officer, Hunter Douglas, Upper Saddle River, NJ, which provides windows coverings. The problem, he says, is with the data, particularly pricing data the vendor builds into the model. "Prices are very negotiable, so the TCO analysis is not realistic," he says. Unless you can plug your actual deal price into the TCO model, you can't get an accurate result, but you don't really know all the final costs until you commit to the deal. It's a classic Catch-22.

Behrad Talebzadeh, director, information systems at Aventis Behring, a biotech company based in King of Prussia, PA, effectively used TCO to justify an enterprise migration from direct-attached storage (DAS) to a SAN-it showed that savings over time would more than pay for the initial outlay-but he too is skeptical about relying on a vendor-provided TCO model to select among products. "A lot of the TCO models are sales gimmicks. It is very hard to benchmark or validate those numbers," he says.

The potential for abusing TCO led IDC to drop TCO analysis altogether. "We haven't done a TCO analysis for storage since the early 1990s," says Bill North, director of research for IDC's storage software service. The problem, he continues, "is the difficulty in trying to quantify a lot of the factors that go into TCO." Too much is left to interpretation.

However, storage managers are about to get deluged with TCO models when Gartner releases its latest storage TCO model and vendors start showing up on the doorsteps of IT managers waving their own Gartner-endorsed storage TCO models.

TCO describes the full cost of owning and operating a piece of technology over a given period of time. It looks at the cost to buy the product and the cost of any accompanying service and maintenance agreements. It then estimates costs for the labor required to support the product, the amount of floor space and energy it consumes, and a host of other factors. It also factors in such issues as capacity utilization, downtime and more.

Now, however, ROI-not TCO-has become the metric of choice. ROI, which attempts to quantify in today's dollars the value of the future payback an organization will likely experience from a technology acquisition, has come into vogue. "TCO is part of the cost side of the ROI analysis," says Mike Schroeck, partner-in-charge, PwC Consulting's iAnalytics practice.

The new Gartner storage TCO model promises to shake up the usual approach to TCO. Like conventional TCO it looks at a dozen or more factors grouped into three main cost components: capital costs, labor costs and service costs, (see "Gartner TCO cost factors") but this time around Gartner is taking TCO beyond basic cost analysis. It's presenting the TCO model as a tool for managing the storage operation rather than simply for comparing vendors or justifying technology purchases. "The model looks at the various business processes that drive storage and builds in some ROI," says Craig Stanley, research director for Gartner's enterprise storage management practice. The Gartner TCO model also looks at service delivery, class of service and storage forecasting-things not typical of TCO models.

Comparing vendors

The Gartner model likely will make vendor comparisons although the model in its current version deliberately doesn't lend itself to such comparison. This proved somewhat contentious in the Gartner-led sessions where the details of the model were hammered out. "We wanted to do a comparative tool, but the big vendors pushed back," says Gabe Gotthard, vice president of 3PARdata.

The problem with vendor comparisons, says Roger Schwanhausser, IBM Global Services' director of the data and storage infrastructure practice, "is the difficulty of holding all the factors constant so you can make apples-to-apples comparisons." In addition, he says, TCO should be only one of many factors in a purchase decision.

Gartner TCO cost factors

Capital costs (fixed, leased or depreciated assets): hardware, software, infrastructure (hubs, switches, routers, directors, etc.), facilities and high-availability costs

Labor costs installation, operation, management, performance, security, capacity planning, evaluation and support of storage technologies; and finance and administrative costs associated with procurement, budgeting, asset tracking and outsourcing contract management

Services costs disaster recovery contracts and expenditures, outsourcing services

With that at least IT managers and the vendors agree. "TCO is one of many factors we use in making storage decisions," says Michael LaPorta, senior project manager, TXU Energy, Dallas. The biggest factor at TXU is how well a given storage technology meets the company's needs. "Neither acquisition cost nor TCO is a primary factor if the product does what we need."

Similarly, at the Parker-Hannifin Control Systems Division, Ogden, UT, an aerospace manufacturer, storage is such a critical part of the company's IT infrastructure that need-not TCO-drives purchase decisions. "When it comes to things like backup, TCO isn't pertinent. At best TCO is most helpful when making budget projections," says Jack Norton, systems analyst.

So many TCO factors are open to interpretation that vendor comparison indeed can be difficult. "How you allocate management costs is important. A slight change in the management cost makes a big difference in the final result," says Russell Fellows, strategic marketing manager at Sun Microsystems. "We claim a low TCO due to our efficiencies of management," he adds. 3PAR, too, claims low management costs, less than one-quarter those of big enterprise storage vendors.

Vendor modifications

Gartner bills the model as vendor-agnostic, but most IT managers won't likely ever see a truly vendor-neutral version. "We're releasing a version for vendors. They can put in their own costs," says Gartner's Stanley.

That certainly opens the model to the potential for vendor manipulation although Stanley says the model cannot be readily tampered with. "Vendors could downplay some features or show 99.999 uptime that users aren't likely to achieve, but this model is not easily subject to this kind of thing," he says. (See "Manipulating TCO models") To guard against egregious vendor manipulation, the model uses Gartner's own data, which can't be altered by a vendor, as the baseline. If a vendor's permitted changes go too far, the model would reject them, Stanley says.

CIOs focus on hardware costs when thinking about TCO

1.  Upfront cost of storage devices

2.  Cost to scale storage

3.  Cost to switch architecture

4.  Cost of storage software

5.  Cost of people to administer storage

6.  Cost of implementation

Upfront costs are the most visible during rapid data growth

Despite being the largest cost, people costs are less visible because they are hidden with cost of administering servers

Source: Merrill Lynch Reality Check Survey, 2/2001.

"We will adopt [the Gartner model] without abandoning our regular approach," says IBM's Schwanhausser. He doesn't expect to modify the model other than to change how it summarizes and presents the results.

StorageTek, Louisville, CO, expects to modify the model only as its pricing and environmental factors change, reports Pete Koliopoulos, director of product marketing. "We wanted the flexibility that things could change over time," he says.

3PAR also expects to make modifications. "We will add in things based on our capabilities," says Gotthard. Thin provisioning could be a huge factor, reducing the amount of storage initially required by a factor of two or three, but it is not in the model," he says. But you can bet it will be in 3PAR's version.

The model mainly gives vendors their say in the qualitative sections, particularly the gap analysis part, says Kristen Newton, senior manager/analyst relations at Network Appliance Inc. "Each vendor will be able to make their own case," she says.

Beyond TCO

Storage managers may find the new Gartner storage TCO model with its emphasis on aligning storage with business requirements more useful than straight TCO. The new Gartner model is still intended to reveal the cost of running the storage environment, but in the context of "the ability to deliver services that, at a minimum, meet the needs of the enterprise," says Stanley. Only by measuring the levels of services being delivered by the data center and comparing them with the total cost to deliver that service level, Stanley contends, can organizations balance cost efficiency with service delivery.

To do this, the Gartner models calls for identifying and segmenting service levels by class of service (COS), since not all applications require the same levels of storage. Gartner defines a COS as a common set of storage services that are delivered to meet a corresponding set of storage requirements. "Once COS' are determined, the level of business requirements and the level of services delivered must be evaluated," Stanley says. The Gartner model then calls for gap analysis to identify differences between expectations and services actually delivered. For each COS, the manager must assess such factors as security, performance, connectivity, scalability, availability, recoverability and manageability. The Gartner TCO model also asks storage managers to forecast future storage needs and understand utilization and their storage-per-administrator ratio-key factors in lowering storage costs. Eventually, the model will direct managers to estimate future cash flows required to support the various solutions offered by vendors to finally arrive at the storage ROI, in effect merging TCO into ROI.

CIOs focus on hardware costs when thinking about TCO

These costs are often overlooked in TCO analysis:

Storage management

Capacity planning/allocation

Configuration changes

Performance evaluations


Data protection


Cost of planned/unplanned downtimes

Cost of data recovery

Costs of business damage

Clearly the model asks a lot of IT managers in terms of providing initial data-data not readily available in most cases. "It is even hard to get a basic inventory of the storage assets," says Schwanhausser, and the Gartner model asks for that and much, much more. Faced with a massive data collection effort, many managers will opt to accept the vendor defaults built into the model.

The Gartner model goes beyond the initial modest aim of TCO-how much will this piece of technology cost me to own and operate over some period of time-to provide a blueprint for managing an enterprise storage environment. It still, however, suffers from the ever-present danger of questionable data (garbage in, garbage out) and the potential for introducing vendor bias, no matter how well Gartner tries to guard the model from manipulation.

It's not clear if IT managers will flock to this new TCO model. The early reaction is less than enthusiastic. "Managers aren't really asking us for TCO," says Esposito. Whether they will once they get a whiff of the new Gartner model remains to be seen.

Web Bonus:
Online resources from "Storage management hassles and headaches" by Giao Tran.

Dig Deeper on Data storage strategy