Valuable data ignored

Splunk has released research that shows organisations are ignoring potentially valuable data and don’t have the resources they need to take advantage of it. The research reveals that although business executives recognise the value of using all of their data, more than half (55 percent) of an organisation’s total data is “dark data,” meaning they either don’t know it exists or don’t know how to find, prepare, analyse or use it.

Wednesday, 1st May 2019 Posted 5 years ago in by Phil Alsop

The State of Dark Data Report, built using research conducted by TRUE Global Intelligence and directed by Splunk, surveyed more than 1,300 global business managers and IT leaders about how their organizations collect, manage and use data. In an era where data is connecting devices, systems and people at unprecedented growth rates, the results show that while data is top of mind, action is often far behind.

· 76 percent of respondents surveyed across the U.S., U.K., France, Germany, China, Japan, and Australia agree “the organization that has the most data is going to win.”

· 60 percent of respondents said that more than half of their organizations’ data is dark, and one-third of respondents say more than 75 percent of their organization’s data is dark.

· Business leaders say their top three obstacles to recovering dark data is the volume of data, followed by the lack of necessary skill sets and resources.

· More than half (56 percent) admit that “data-driven” is just a slogan in their organization.

· 82 percent say humans are and will always be at the heart of AI.

“Data is hard to work with because it’s growing at an alarming rate and is hard to structure and organise. So, it’s easy for organisations to feel helpless in this chaotic landscape,” says Tim Tully, chief technology officer, Splunk. “I was pleased to see the opportunity people around the world attach to dark data, even though fewer than a third of those surveyed say they have the skills to turn data into action. This presents a tremendous opportunity for motivated leaders, professionals and employers to learn new skills and reach a new level of results. Splunk can help those organizations feel empowered to take control of identifying and using dark data.”

Respondents are Slow to Seize Career and Leadership Opportunities

While respondents understand the value of dark data, they admit they don’t have the tools, expertise or staff to take advantage of it. Plus, the majority of senior leaders say they are close enough to retirement that they aren’t motivated to become data-literate. Data is the future of work, but only a small percentage of professionals seem to be taking it seriously. Respondents agree there is no single answer, though the top solutions having potential included training more employees in data science and analytics, increasing funding for data wrangling, and deploying software to enable less technical employees to analyze the data for themselves.

· 92 percent say they are “willing” to learn new data skills but only 57 percent are “extremely” or “very” enthusiastic to work more with data.

· 69 percent said they were content to keep doing what they’re doing, regardless of the impact on the business or their career.

· More than half of respondents (53 percent) said they are too old to learn new data skills when asked what they were doing to educate themselves and their teams.

· 66 percent cite lack of support from senior leaders as a challenge in gathering data and roughly one-in-five respondents (21 percent) cite lack of interest from organization leaders as a challenge.

AI is Believed to Be The Next Frontier for Data-Savvy Organizations

Globally, respondents believe AI will generally augment opportunities, rather than replace people. While the survey revealed that few organizations are using AI right now, a majority see its vast potential. For example, in a series of use cases including operational efficiency, strategic decision making, HR and customer experience, only 10 to 15 percent say their organisations are deploying AI for these use cases while roughly two-thirds see the potential value.

· A majority of respondents (71 percent) saw potential in employing AI to analyze data.

· 73 percent think AI can make up for the skills gaps in IT.

· 82 percent say humans are and will always be at the heart of AI and 72 percent say that AI is just a tool to solve business problems.

· Only 12 percent are using AI to guide business strategy and 61 percent expect their organisation to increase its use of AI this way over the next five years.

Regional Differences

There are some key differences in the UK specific results. For example, 39 percent of people in the United Kingdom believe AI can make up for the skills gap versus only 27 percent globally. UK employees are also the most likely in the world to say they need to learn more data skills in order to get promoted again, 83 percent compared to the global figure of 76 percent. Additional UK specific results include:

· The UK often comes second only to China in its enthusiasm for data and AI, and its belief in the importance of data skills

· 67 percent of UK companies agree “data-driven” is just a slogan at their organization, compared with only 56 percent globally

· The majority of respondents in the UK market (61 percent) report understanding AI extremely or very well — one of only two markets in which a majority make that claim (the other is China, at 77 percent). The global average is 48 percent