This article can also be found in the Premium Editorial Download "Storage magazine: Slimmer storage: How data reduction systems work."
Download it now to read this article plus other related content.
Think about it. Perhaps the first big data analytics will require an interpreter. Early models may be difficult to set up without the skills of someone who knows how to collect certain types of data and compare data meaningfully to yield still-cryptic grist for the empirical grindstone. Think of all those political analysts on 24-hour cable news networks who need to parse all the little bits of data to fill time and make meaningless observations about the present and its potential impact on the future. Data scientists may perform the same role with early big data analytics.
However, if IBM’s Jeff Jonas is correct, and big data analytics will eventually produce algorithms that mimic human reflection to solve increasingly self-generating questions, we won’t need someone to read tea leaves in the machine cup. The results will be set forth as statements of truth or self-generated action plans. The data science guy will be out of work as quickly as the oil lamp guy.
On the other hand, if algorithms do parse data in a reflective way that mimics too well the human mind, we may need a data scientist less than we will a disk whisperer or a data psychologist. We’ll need to help these big data algorithms deal with issues ranging from their upbringing to their psychoses and neuroses. What if the algorithm is used to spot voting fraud, only it finds no instances of voting fraud and begins to conclude that its mission is a cruel, cynical, politically motivated hoax
Sounds pretty complex. The alternative might be that we need a lot more competent programmers, system engineers and database admins. Only those names don’t sound quite as sexy as “data scientist.”
BIO: Jon William Toigo is a 30-year IT veteran, CEO and managing principal of Toigo Partners International, and chairman of the Data Management Institute.
This was first published in October 2012