|The ultimate benefit of consolidation|
"The SAN island problem is the same problem as separate server-attached storage, just one step higher up," says Bob Venable, manager of information systems, Blue Cross Blue Shield of Tennessee (BCBST), in Chattanooga. The health insurer is concerned that multiple SANs hinder enterprisewide storage sharing.
Experts generally agree that consolidating multiple DAS units into fewer, larger storage subsystems reduces the cost of ownership and simplifies maintenance and administration through centralized management. They don't all agree on how quickly that will occur (the payback period) or whether that should be your real goal (see "The ultimate benefit of consolidation," on this page). The most immediate question, though, is how much to consolidate and how best to achieve it.
The generally recommended approach to storage consolidation is networked storage, which gives multiple servers access to multiple storage devices.
Industry research firms are nearly unanimous in predicting that SANs will become the dominant enterprise storage format within the next few years.
So, how should you organize your SANs? Should you have dedicated SANs based on factors such as geography, application, platform, corporate organizational structure and function or one huge enterprisewide SAN?
The case for multiple SANs
The correct architecture for you will depend on your applications, the diversity of your environment and a host of nontechnical issues that could roughly be grouped under the heading of "organizational issues," or to put it more bluntly, "politics."
Ideally, if you're consolidating storage, "one large monolithic SAN is preferred," says Arun Taneja, a consulting analyst with Taneja Group, in Hopkinton, MA. As a practical matter, implementing multiple SANs may be unavoidable, says Dave Hill, vice president of storage research, Aberdeen Group, Boston. "In large companies, you can't consolidate all the storage at once," Hill contends. "The Holy Grail is a single storage infrastructure, but there are practical reasons for multiple SANs. The question is: What level of aggregation do you need or want?"
The most obvious path to follow is to create multiple SAN islands, each consolidating a portion of your servers and storage. Despite dire warnings from switch vendors and consultants, that's not a problem, says Tom Black, technology services lead, at Petro-Canada, Ontario. For example, multiple SANs allow for greater flexibility and risk management. "If you are doing something risky, you can separate it on a different SAN," he says.
Hill agrees: "I don't see any problem with islands of SANs. It's a positive development." Different SANs, he explains, can support different functions that don't need to share data. In other cases, the organization may have critical systems and, for security reasons, not want to share the data, or the company may have geographically separate facilities or autonomous business units, each with its own SAN. In situations such as these, multiple SANs make sense.
But "SAN islands are difficult and expensive to manage," says Tony Prigmore, a senior analyst at the Enterprise Storage Group, Milford, MA. The piecemeal approach may not be beneficial moving forward, he says.
How to decide
Experts agree that it's not a matter of a coin toss or taste: The choice of how to deploy SANs should be determined either by the application or by the service level.
"A lot of factors eventually come into play, but we start by determining the level of service in terms of performance and high availability," says Scott Robinson, CTO, Datalink Corp. A high-availability SAN, for example, offers redundant everything, which significantly increases the cost, but "doesn't make sense for everything," he says.
Intuit Inc., Mountain View, CA, is one company that takes this approach, setting up its SANs according to quality of service demands. "Everything that needs five-nine reliability we put on the EMC SAN with director-class switches and high availability," a costly configuration, says Eric Ihde, a senior systems engineer. The less-critical systems are put on SANs that use fabric switches and ATA drives, both of which cost significantly less. The company puts development work--which makes the least demands in terms of availability, but often has the highest risk of causing problems--on a SAN with a small edge switch, the least costly of all.
Intuit's approach is driven by the economics of high availability. "We don't want to spend money on high availability where we don't need it. We want to save the costly directors and high-availability storage for the things that are really worth it," explains Ihde. "The developers don't need 24/7 availability."
Taneja takes a different tack than Robinson and Ihde, looking at the application. "Start with the application. Then you can factor in the other realities," he says. Those other realities include platform considerations, security issues, geography and distance and internal organizational and political dynamics.
"The easiest way to propagate SANs is by application or by department and that is how companies have been doing it for the past three years," says Prigmore.
This is the approach Petro-Canada took, where the platform and the application were, in effect, the same thing. "We set up our SANs based on the platform," explains Petro-Canada's Black. The firm's Unix SAN, for instance, handles the Oracle database, transaction processing and a 1.5TB SAP database. The company has a SAN for its large NT servers, but many of the company's NT systems are widely dispersed and small.
|ISL heaven or hell?|
The middle ground between a single consolidated storage area network (SAN) and multiple SANs is the logical SAN consisting of multiple SANs connected through inter-switch linking (ISL). ISL uses the expansion (e-) port to connect one switch to another to turn multiple SAN fabrics into one logical consolidated SAN. For enterprise-scale high performance, companies can opt for ISL trunking, which uses multiple e-port connections to create one high-speed connection linking multiple switches. ISL trunking, for example, can aggregate the bandwidth of the switched ports, so four 2Gb/s switches can be aggregated, theoretically, into an 8Gb/s pipe, although some bandwidth is lost to overhead.
"Regardless of how many SANs you have, you want to manage them as one," says Mike Jones, vice president, ViON Corp., a systems integrator based Washington, DC. ISL allows you to do just that, manage multiple SANs as one logical SAN.
Storage managers at Blue Cross/Blue Shield of Tennessee (BCBST), however, rejected the use of ISL. "It allows access between the SANs, but it introduces imbalances and problems. It has the potential to become a huge bottleneck. You end up strangled by your ISL," says Bob Venable, manager of information systems, BCBST, in Chattanooga. Certainly, ISL complicates issues such as routing and load balancing, although these capabilities can be built into the ISL device.
Intuit has also rejected ISL. "You would only use ISL if you really have to merge the two SANs. But then, everyone sees everything on each fabric, which raises security concerns," says Eric Ihde, a senior systems engineer for Intuit. Of course, you can use zoning and LUN masking to limit who sees what, but configuring those capabilities gets complicated in a merged SAN. "There is a lot of work you have to do in advance to make sure there are no naming or configuration conflicts. Otherwise the whole thing can break down," Ihde adds.
Ultimately, ISLs can only go so far before they consume significant amounts of ports, making them uneconomical.
Ultimately, the application-based and service level- based approaches actually are quite similar and will generally lead the organization to the same place. Most consultants see one of those approaches as the No. 1 priority, and then look at a variety of factors including security, geography, platform, storage management skills, cost and organizational politics. They try to avoid designs that access primary storage across long distances.
And most feel SAN technology today has progressed to the point where skilled storage managers can confidently restrict access to designated storage pools on the SAN.
For example, if you are looking at mission-critical production applications, you will opt to put all the associated storage--regardless of platform--on a single, high-availability SAN. If the organization is highly decentralized with applications owned and controlled by individual groups, then a SAN for each group is the likely result. With geographically dispersed facilities, there is little choice but to create local SANs to avoid accessing primary storage over long distances, which incurs cost and performance penalties. (See "SAN deployment: Two views")
SAN architectures are clearly going to be moving targets in the coming years. Prigmore suggests that organizations should try to gradually evolve various SANs into a consolidated storage networking infrastructure. One key to that evolution is to create separate SANs on the logical, not the physical level, where possible. Prigmore advocates multiple logical SANs that will run over a single physical SAN infrastructure. Storage managers can use zoning and LUN masking, both of which end up restricting access to parts of the SAN infrastructure, to set up the logical SANs.
Zoning and LUN masking may evolve into more flexible structures, like Cisco's Virtual SAN technology. This technology (see "Virtual SANs put to the test" in the July issue of Storage and "Getting remote data right," also in the July issue) applies constructs very similar to virtual LAN technology to segregate ports within and across switches to specific SANs. Ultimately, Cisco's approach--or something like it--could become a standard in the near future, making it possible to apply the technology to switches from different vendors.
Less is more
Regardless of how many SANs a company operates, ideally it still wants to manage all the storage operations with one tool from the fewest locations possible. How many SANs you run matters from the standpoint of cost and efficiency. More SANs cost more--although the cost difference is small--and are more complex to manage.
"There is a slight incremental cost for each additional fabric. And each fabric needs to connect to SAN management devices and its own console," says a storage manager for Factiva, a Dow Jones and Reuters Company.
When Factiva launched its Factiva.com portal, it deployed six HPQ StorageWorks SANs in two geographically separate data centers managed by HP's SANworks management appliance. The official number of SANs, however, is a bit deceptive. "I don't consider it six SANs. Depending on the semantics you use, it's really more like three SANs and six fabrics," says the storage manager who spoke under condition of anonymity.
Under the best of circumstances, he says, Factiva would have two SAN fabrics in each of its two data centers to ensure high availability. The company never considered running one pair of SAN fabrics across both data centers.
Companies that want to implement a single consolidated SAN need to consider the practical limits to current SAN scalability, such as performance and complexity. "You can't answer the size question easily," says Prigmore. Most SAN islands support about 10 servers. Although a few large SANs contain thousands of ports, those are rare. "IT will just have to decide where it wants to draw the line from a management and security perspective," he says.
BCBST doesn't see scalability as an issue. "We can support 100TB of storage with the full-time equivalent of one and a half people and could easily go to 200TB," says Venable, who has hundreds of servers attached to that SAN. The company--which is consolidating storage on a SAN consisting of seven IBM ESS storage arrays, McData fabric switches at the edge and two mirrored McData director-class switches--doesn't see a practical limit to how far it can scale its consolidated SAN. To manage the consolidated storage, it uses McData's SAN Navigator management software.
The bigger danger, in the view of the managers at BCBST, isn't scalability as much as starting too small. "We look at where we want to be in five years and start heading there now. We don't try to get there in lots of little steps," explains Hugh Hale, BCBST's director of technical services. The company's five-year plan was to get to consolidated enterprise storage sharing and get away from islands of storage. To date, the company has brought its 25 Unix and 250 Windows servers into the consolidated SAN. Soon, it will pull in the mainframe.
Ultimately, there's no simple answer to how large a SAN can or should grow. "The only thing I'd say is that the limit gets higher every year," says Hale. The size of any given SAN, ultimately, is constrained by any number of factors, ranging from available ports to bandwidth to management capabilities.
Much the same can be said about the question of whether to consolidate SANs or operate multiple SANs. Many factors will go into the answer. "Maybe we should consolidate our SANs eventually," says Petro-Canada's Black. But in the meantime, there are many organizations which have valid reasons for continuing to deploy multiple SANs, while others have equally valid reasons for consolidating them into one SAN. Do SANs taste great or are they less filling? You'll have to decide which is most important to you.