- Scott Sinclair, Enterprise Strategy Group
With the rise of the digital economy, businesses are redefining themselves in part or wholly as software companies. While IT has been part of business for decades, its role has changed. Data and technology are now core elements of modern business strategy.
New digital initiatives, often categorized as digital transformation, aren't simply about using data to run your business. You're already doing that. Digital IT transformation is about using data and IT services to gain a competitive advantage. For instance, you can use analytics to become more efficient, to better understand your customers or to create digital products and services that open new revenue streams.
The critical component of digital IT transformation is the deployment of emergent transformational workloads, such as business intelligence and analytics, or software that collects data at the edge of the enterprise as part of internet of things initiatives. These workload categories are transformational and will continue to grow. But learning technologies, such as artificial intelligence and machine learning, are gaining traction and may ignite the next wave of IT investment.
AI vs. machine learning
There are several definitions of what constitutes AI and machine learning workloads. AI is considered the more general term, related to the integration of intelligence into software. Machine learning -- also known as deep learning -- involves methodologies used to generate software intelligence.
At a high level, machine learning is used to develop algorithms that use data and statistical analysis to predict an output, such as looking at your shopping history to recommend something to buy. With learning though, instead of writing the algorithm, you train it through learning. Typically, these workloads require a tremendous amount of processing and high-performance storage access to data when training the algorithms.
So how are learning initiatives different from other digital IT transformation initiatives? It depends on the company and the industry, but generally with machine learning, there's a direct correlation between the amount of data used and the quality of the algorithm created. When algorithms are used to accelerate revenue generation or open new revenue streams, more data is collected, which enables the creation of superior algorithms. This can give early adopters a significant competitive advantage.
A recent Enterprise Strategy Group research study found that 38% of organizations with active AI and machine learning initiatives said they're depending on these projects to deliver significant, measurable business outcomes immediately. In other words, for companies investing in these technologies, AI and machine learning often are the business, with a significant portion expecting immediate returns.
The priority being placed on AI and machine learning has shown up in hiring over the past several years. The interest in AI and machine learning expertise is so prevalent that Glassdoor identified data scientist as the best job in America three years running.
Learning technologies: The next step
The next step for organizations investing in AI and machine learning is to provide personnel with the right tools and infrastructure. Here's where issues occur, however.
When Enterprise Strategy Group asked decision-makers familiar with their companies' machine learning and AI initiatives to identify what barriers hold projects back, the narrative shifted to the infrastructure. For example, 44% of organizations with active AI and machine learning projects said IT infrastructure in the form of cost or lack of capabilities was one of their top three challenges with these initiatives. The lack of sufficient infrastructure capabilities is a major concern.
The following recaps what we know about AI and machine learning:
- A significant percentage (38%) of companies that have these initiatives underway expect them to deliver significant, measurable and immediate business outcomes.
- These organizations are investing heavily in personnel.
- The inherent nature of the AI and machine learning workloads offer significant competitive advantages for certain industries. Better algorithms improve business results, which lead to more data, which results in better algorithms and so on.
- Infrastructure holds back nearly half (44%) of organizations surveyed.
It's that last bullet, the complexity of infrastructure, that I expect the IT vendor community will address quickly. Part of the complexity may have to do with using of graphical processing units or field-programmable gate arrays to accelerate compute and manage highly performant and often highly parallel storage behind all that processing. IT vendors have already started working to resolve concerns. Nearly every major storage vendor offers reference architectures for AI and machine learning workloads. For example, Pure Storage, in a partnership with Nvidia, recently announced its AI-ready infrastructure product.
Ultimately, I expect data-driven businesses using AI and machine learning workloads to pursue digital IT transformation will quickly wake up to the challenges of infrastructure and start addressing them. And if you're just starting with AI and machine learning, it's important to understand the impact the infrastructure can have on your initiatives.
How difficult is it to configure and set up? How does the performance affect the quality of the outcomes? Even in small environments, it's important to consider who will manage and configure the infrastructure?
Often, as initiatives get started, you can manage the hardware outside the traditional IT organizations. While understandable, these ownership decisions can drive up the personnel costs, such as when highly paid data scientists end up spending considerable time managing hardware. These workloads and the digital IT transformation they engender are simply too important, and businesses have too much invested in personnel, to let infrastructure serve as a stumbling block to success.
- Meeting the Storage Needs of AI –BMC Software