In a virtual server world, the concept of LUN technology and the amount of attention LUNs require from storage administrators will soon be a thing of the past.
The era of LUNs and volumes, as we have known them for decades in the data storage industry, is quietly coming to an end. And if you ask me, it's for all the right reasons, even if storage administrators may feel threatened by the change.
In the world of physical servers, our standard practice of ganging up a set of disk drives into a RAID set to create a LUN has served us very well for decades. This LUN was created recognizing the type of application it was to serve and it was associated with all the appropriate storage services (replication, compression, snapshot and so on) the application warranted, based on its importance. That's all well and good. But then we started servicing several applications from the same LUN, and unless we overdid it or the applications were erratic, the LUN was able to serve multiple applications. If an application was important enough it got its own LUN and associated services, even if the utilization of either capacity or performance was sometimes less than ideal.
But then server virtualization came along and all hell broke loose. One or a few LUNs serving a multitude of virtual machines (VMs), maybe even several hosts, each with tens of VMs or more representing a variety of applications/workloads, simply couldn't cut it. The infamous I/O blender effect is now well understood. The precisely tuned LUN of yesteryear was wrestling with totally random I/O coming in an unpredictable onslaught from a large number of VMs. The storage controllers were overwhelmed and application performance plummeted.
The storage industry responded with a variety of solutions to this problem. The traditional storage array vendors added solid-state drives to deliver more I/Os. They also virtualized the arrays internally so all disk drives would participate in serving all VMs, rather than only a few drives getting hit constantly. And they added features to more tightly integrate their systems with VMware or Hyper-V using application programming interfaces such as vStorage APIs for Array Integration, vStorage APIs for Storage Awareness or Hyper-V's Offloaded Data Transfer. We saw a new crop of vendors come to market with storage built from the ground up for the virtual world. That list includes Gridstore, Nimble Storage, Nutanix, Scale Computing, SimpliVity, Tintri and others. Hewlett-Packard responded with its StoreVirtual VSA, presented either as a VM to be used with any storage or as a complete appliance offering. Yet other newcomers developed software that worked with existing storage arrays but delivered an order of magnitude improvement in performance, latency and capacity reduction. The classic example was Virsto Software, which was acquired by VMware.
Those latter two product categories have taken a very different path to delivering storage services to applications. They are 100% VM-centric. They completely get rid of the LUN-centricity that storage shops have been addicted to since the beginning of the SCSI era. Via policy, one assigns the level of importance to a virtual machine. That policy establishes the type and amount of storage the VM would receive, where that data would be placed, how many times it would be replicated and what type of data protection it would receive. In many such offerings, the VM can even be assigned a Quality of Service (QoS) to ensure it has priority to receive the appropriate resources if there's contention. These offerings are monitored on a virtual machine basis and deliver performance data, capacity utilization and other relevant information on a per-VM basis. You don't back up a LUN, you back up a specific VM. You don't clone a LUN, you clone a VM. You don't manage storage from a storage console, you manage it from VMware vCenter or Microsoft System Center.
The server administrator becomes the person in charge of provisioning storage at a VM level, not the storage admin. There are still some management tasks that require the storage administrator's deep understanding of how storage works under the covers, but day-to-day storage management now shifts to the server administrator.
Activities such as creating new LUNs, tuning LUNs for changing conditions of applications, choosing the right RAID groups, and replicating volumes synchronously or asynchronously are no longer needed. A lot of those functions have been automated and are triggered (you guessed it) at the VM level. Performance metrics, including those we've historically associated with storage, are now viewed from VMware vCenter or Microsoft System Center VMM at a VM-level granularity. Working at a LUN level is simply not meaningful anymore.
Of course, there are differences in the way the various products mentioned here have implemented these functions. Products from Nutanix, Scale Computing and SimpliVity are more than storage; they're infrastructure in a box. Nutanix and SimpliVity only support VMware; Scale Computing only supports KVM today. Regardless, the storage is 100% VM-centric and you couldn't find a reference to a LUN if your life depended on it. Tintri is storage for VMware only today, while Nimble Storage supports all major hypervisors. Gridstore implements a virtual storage controller as a miniport driver for Hyper-V. It uses the virtual controller's awareness of each application's assigned priority and I/O needs to enable sophisticated QoS and auto-tuning functionality. But despite these differences, these products are conceptually bound by their VM-centricity.
To be sure, the world has not fully transformed yet. Legacy storage players are burning the midnight oil to make their array capabilities work more effectively with VMware, Hyper-V and XenServer. They're also surfacing storage management information at the server virtualization admin's console. But there is no doubt that the era of cumbersome LUN management is over. LUNs may not be dead under the covers, but they're certainly dead as the main weapon wielded by a storage administrator. LUN creation and selection is much more easily done by the storage system, working in conjunction with the application and the hypervisor, in a totally automated fashion. Doing things at a VM level, by definition, provides application awareness -- knowledge that can be used for a vast number of purposes, such as load balancing, auto-alignment and delivering the right QoS.
Storage administrators can now be relieved of all the mundane chores of delivering storage to applications and instead focus on strategic matters, such as infrastructure planning, analyzing information to improve efficiency and the like. Of course, data protection, archiving and disaster recovery still need to be dealt with, as do all things related to cloud storage. It might take the industry another three years to get there but the handwriting is on the wall. A smart storage administrator is already preparing for the day when life will be LUN-free. LUN technology has served us well, but it's time to put LUNs to rest.
About the author:
Arun Taneja is founder and president at Taneja Group, an analyst and consulting group focused on storage and storage-centric server technologies.