Storage vendors could learn a thing or two from the Redmond gang about plugging holes in leaky systems.
When I joined ESG in 2003, the acronym stood for Enterprise Storage Group. As the company diversified into areas like security, we decided we needed a new middle name. Goodbye "Storage," hello "Strategy" and the Enterprise Strategy Group.
Why am I sharing this story? ESG was a leading storage analyst firm, but fairly unknown outside the spinning disk and magnetic tape crowds. When I joined ESG to start the information security practice, I decided that one of my initial focus areas would be security issues as they relate to storage. ESG had great relationships with storage professionals and vendors, and I found little actual research or writing about storage security. It seemed like a green field, so off I went.
After diving into storage security, I quickly realized why so little had been written on the subject. Storage professionals assumed security was something the networking and server folks had to deal with--a true case where ignorance was bliss. With no user demand for storage security, vendors were more than happy to snub security and concentrate on standard storage functionality around performance and availability.
The frightening world of storage security
Given that no one was truly minding the storage security store, I found a scary situation full of holes the size of Volks-wagens. Storage vendors seemed to eschew security in the way they designed, built and managed their products. Few field engineers, developers or chief technology officers ever mentioned security in customer meetings or analyst briefings. I discovered that almost all storage technologies:
- Were never tested for software-based security vulnerabilities. Storage software was filled with insecure interfaces, unnecessary functionality and buffer overflows. The code wasn't even a challenge for script kiddies, let alone sophisticated hackers.
- Left management interfaces wide open. This was especially alarming because hackers often rely on scanning networks to map IP addresses, discover hosts and find open applications. Storage devices were "sitting ducks."
- Had few processes for security bug tracking and patching. In the storage world, software updates were designed to repair software functionality glitches and came out a few times a year. Few vendors had anything in place to monitor, test, fix and distribute patches for addressing security vulnerabilities.
- Relied on insecure channels for storage management. Many storage professionals logged onto storage management apps over insecure protocols like Telnet and HTTP, rather than HTTPS or SSH. Critical storage management data was transported willy-nilly around the network in cleartext.
- Depended on basic authentication. Changing configurations on enterprise-class storage systems required only a user name and password combination. Even scarier, most users would simply log onto devices as "admin," gain root access to the systems and have the ability to change anything.
- Didn't log events. Storage systems may have had some proprietary logging format, but few users knew about it, let alone turned it on.
- Had insecure default configurations. Storage was insecure by default, meaning that if you configured an enterprise storage system the way the vendor recommended, it was wide open to the bad guys.
I was pretty shocked at the time and decided something had to be done. In March 2004, I responded with a storage security manifesto that outlined 42 security-related product features, storage architecture considerations and processes users should demand from vendors. I was proud of this effort until I realized that I was the only one who thought it was valuable. Storage professionals and vendors scoffed at the notion that security was an issue. Everyone went on their merry way while I became the storage industry equivalent of Chicken Little.
But things have changed. Between the string of visible data breaches and regulatory compliance, the storage crowd woke up to the fact that security was a requirement. As storage professionals put the heat on vendors, the industry reacted. Network Appliance (NetApp) bought Decru. The Storage Networking Industry Association pushed security protocols. EMC grabbed RSA Security.
These were all positive steps, but there was still something lacking, namely a visible commitment to security. I was always longing to see a storage company that would integrate security into its internal culture and make it a pervasive part of the product design, code development and customer-support processes.
The storage industry should learn from Microsoft
Storage professionals and vendors could learn a lot about embracing security from an unlikely source: Microsoft. Back in the late '90s, there was nothing but bad blood between the security community and Microsoft, so much so that Windows became the go-to target of the computer underground. Viruses and worms like the Melissa virus (1999), Code Red (2001), Nimda (2001) and SQL Slammer (2002) were wreaking havoc on Windows, Outlook, Exchange, IIS and SQL Server. Something had to be done.
Most people attribute the turnaround in Redmond to Bill Gates' January 2002 email describing the need for Trustworthy Computing, but security was already turning the corner at Microsoft. Security tiger teams worked with product groups to help them with secure code design, development and testing, a process later formalized as the Security Development Lifecycle (SDL). Today, every new product must go through the SDL process. SQL Server 2005 was the first product to pass the SDL hurdle and it shows. The number of post-development security vulnerabilities is considerably lower than in previous versions. The new desktop OS, Windows Vista, also went through SDL.
Microsoft added security to a number of internal processes. It changed the way it responds to software vulnerabilities by tightening processes, fixing all problems for all software once per month and reaching out to customers with proactive communications. Microsoft also became more serious about its own security tools and technologies. The company enhanced homegrown technologies like its Internet Security and Acceleration (ISA) Server, Microsoft Operations Manager (MOM) and Active Directory (AD) while it went on a buying spree, scooping up security vendors such as Giant Company Software (anti-spyware), Sybari Software (email security) and Whale Communications (SSL VPN).
Microsoft achieved a complete turnaround on security between 2001 and 2007. Its development and support models are now highly regarded in the security community, and its security products are gaining share and becoming market leaders.
Is anyone in the storage industry even close to pulling this off? I do see a number of leading indicators that the industry is moving in this direction. For example:
- NetApp/Decru is championing an effort around key management, building a development community and pushing industry standards.
- IBM extended mainframe security functionality such as its encryption facility, Integrated Cryptographic Services Facility (ICSF) and Resource Access Control Facility (RACF) to storage management software and devices.
- In a model similar to Microsoft's, EMC has instituted its Common Security Platform (CSP), a set of security requirements for all EMC products. CSP covers everything from the way products are built to access controls to logging.
- Hitachi has embraced the ISO/IEC 21827:2002 Systems Security Engineering–Capability Maturity Model (SSE-CCM) to introduce security best practices in product development projects. Security testing has also transitioned from an ad hoc process to a formal phase in the QA cycle.
The bottom line
Security is neither a product feature nor a sound bite for marketing pitches. It's a cradle-to-grave commitment that spans products, processes and personnel. Microsoft proved you can turn on a dime if you want to, and several leading storage vendors are following its example. Storage professionals should be wary of any vendor that hasn't made this transition.