GlassHouse Technologies, Inc.
Published: 13 Sep 2004
Application owners often complain about the time the storage group needs to respond to requests for additional storage. Users may overstate storage requirements to ensure that their actual and potential needs are met. One dramatic example is a financial firm where 5TB of tier-1 storage was allocated for a data warehouse that was currently using less than 100GB. The allocation was based on projected capacity needs three years out!
Provisioning goes beyond carving out disk space and creating and presenting LUNs. It's a multilevel process that requires well-defined procedures. (See "Users slow to embrace storage automation")
Let's examine some of the prerequisites for an effective provisioning operation.
Classes of service. The first prerequisite is a clear definition of each service level's attributes to provide a foundation of common understanding between business units and the storage group. Ideally, class of service is backed by a financial model, so business units can choose service levels that match their business needs. Defining a class of service is more than specifying a type of primary storage--other attributes may include availability, performance, scalability, backup, recovery, archiving, disaster recovery (DR) and security.
The request process. How business units request storage provisioning must be a documented process subject to organizational policy with explicit expectations of service. The procedure should define workflow and authorization along with performance metrics and completion artifacts.
Document the environment. Provisioning depends on discovering and measuring asset utilization of the entire environment. This includes arrays and controllers, storage network components, tape subsystems, media and management servers, links for replication and recovery site storage. This also includes documenting host initiators, multiple data paths, switches, routers, fan-out/fan-in ratios, interswitch linking, management LANs and associated firewalls, array controllers, etc.
Understanding capacity. Provisioning doesn't stop at primary storage. When a gigabyte is requested, five to 30 times that amount will be needed in secondary storage, depending on backup and archiving requirements. This may affect network traffic to backup servers, servers' processing loads and tape subsystems.
DR, particularly for top-tier applications, may mean additional requirements. The requested storage and its data change rate can affect synchronous and semi-synchronous links and may require storage at the recovery site.
Determine utilization. Another prerequisite is knowing the capacity that's actually available. This isn't just the storage requested, but also the fabric consequences of adding primary storage. Traffic, dual paths and fan-in/fan-out ratios can all be affected by provisioning new storage. Utilization must be carefully defined; databases and volume managers grab chunks of space that appear to be used, but aren't fully utilized.
Interoperability and standards. A number of standards have to be in place. To some extent, standards are required because heterogeneous storage and SANs aren't fully interoperable. They're also required to identify default LUN size and for mapping initiators to LUNs to protect the fabric's integrity. Depending on class of service, standards are required for dual-path capability, acceptable fan-out/fan-in ratios, use of interswitch links and port connectivity on the array.
|Provisioning: six levels of control|
Developing provisioning procedures
With the prerequisites in place, a standard operating procedure (SOP) can be developed for provisioning storage in each class of service, on each storage technology and within each storage fabric. Write the base procedure first and then add details.
The SOP includes these major activities:
Processing the request. The first step is a review of what's required by the business unit. The following must be provided in the request:
- Requestor ID and authority. The provisioning request must be authorized based on company policy and should contain identification and authorization data, such as business unit name, contacts, project information, etc.
- Requested action. Requested actions may include new storage for a new application or server, additional storage for an existing application or server, reduction or deletion of allocated capacity or change of service level.
- Provisioning requisition. This section states the amount of data, class of service and implementation date required and the applications or servers that need to access the data.
Planning and design. The planning process assumes that class of service decisions have been made, and that standards are in place, as well as tools and techniques to enable discovery, visualization and traffic analysis. The planning process has three key activities:
- Mapping. Mapping identifies the components through which the newly provisioned data will pass. If new fabric or network is required, the business unit should be notified and timeframes negotiated for acquisition of additional infrastructure. The mapping component should include primary storage, backup storage, archiving storage and DR storage.
- Bandwidth analysis. This involves assessing the impact of additional traffic that the request will generate. Current traffic on various components of the environment must be identified, as well as the current and projected loadings that the new data may trigger.
- Optimizing design. Gaps in capability vs. projected needs must be addressed by adding to, reconfiguring, upgrading or otherwise redesigning the storage fabric and its components.
It's important to capture and document this process. The design process must also identify new components that are required, including a formal process that proves the components meet standards. Once the design has been validated to meet standards, class of service requirements, and compliance with policy on traffic, connectivity and failover, the design can be submitted for formal review and approval by a peer committee.
Implementing the request. Provisioning often involves groups outside the storage staff. For example, installing HBAs might be a system administration function. Development staff, DBAs and business analysts may also need to be involved. (See "Provisioning: six levels of control")
There may come a time when these activities can be captured in an application that will manage the workflow, notify the appropriate individuals and even perform the configuration activities. However, that day isn't likely to be here soon. Defining and developing the standards, tools and techniques outlined here is absolutely key to an orderly, low-risk provisioning process.
- Tiered Storage - Optimizing the Storage Infrastructure –Fujifilm Recording Media USA, Inc.
- Illuminating Insight for Unstructured Data at Scale –IBM