Analytical mistrust hampers data-driven agility

Datawatch  has published the results of a new research report from TDWI that finds company executives do not trust their internal data and analytical processes for effective business operations.

According to the Datawatch-commissioned survey, "Reducing Inefficiency and Increasing the Value of Analytics and Business Intelligence," only 11 percent of respondents said that they were very satisfied with their companies' investments in data and analytics projects to meet strategic goals for enabling data-driven decision-making or actionable customer intelligence.

"Data is the lifeblood for critical business activities including risk evaluation, customer engagement, business performance management, regulatory requirements and more. When business users cannot access governed data, share it and collaborate on analytical outcomes, they are left feeling frustrated and ineffective," said David Stodder, senior director of TDWI Research for business intelligence (BI). "Business users need to move beyond spreadsheets and inefficient data preparation practices to a team-based intelligent analytical approach that enables smarter data stewardship."

The survey of 263 business and IT executives exposes the data management and governance shortcomings of many organizations and the reverberating business impact of poor data quality, lack of confidence and the scarcity of collaborative frameworks. Key findings from the survey include:

• Spreadsheets continue to be the preferred method for analytics. Eighty-nine percent of respondents state that they input data into spreadsheets for analysis, yet 81 percent are concerned about data quality and consistency when using these files.
• The majority of those surveyed (65 percent) say that users and analysts can create analytical dashboards with self-service applications; however, only 44 percent can find or access relevant data for the reports.
• One in five (20 percent) responded that individuals can find trusted data sources without IT support, and only 18 percent can track data lineage to a source, which leads to a decay in analytical confidence.
• Inefficiency related to data preparation tasks for analytics continues to be an issue with nearly half reporting (48 percent) that users are spending at least 61 percent of their time on finding and preparing data.
• Tribal knowledge about data use and sharing sources is conducted in less formal, governed formats, including email (48 percent), word-of-mouth suggestions (45 percent) and internal and external networks (25 and 5 percent respectively). Few organizations (5 percent) are using data marketplaces for collecting this knowledge and curating trusted data sources.
• Fifty percent of respondents stated that their organizations do not have a formal data governance strategy, which directly impacts how data usage is tracked for a complete data lineage. This lineage is key to increasing trust in data; yet nearly two in five (38 percent) are only somewhat confident about the lineage of the data used in reports and analytics, and 18 percent report no confidence.
• Data stewardship and sharing insights is essential to increasing trust in data, assisting in enforcing data governance policies and expanding collaboration across the enterprise, still only 36 percent of respondents find that data stewards are helping with the selection of data sets. Additionally, a small amount (22 percent) indicate they share feedback or rate the analytical outcomes shared across the company.

"While many organizations talk about how they want to leverage data for operational processes, business insights and competitive purposes, the reality is many executives are leery of using analytical outcomes when they cannot be confident of where the data came from and how it was analyzed," said Ken Tacelli, chief operating officer, Datawatch. "This disconnect has a profound impact on a business' bottom line. A data intelligence framework can help organizations avoid these pitfalls and allows for team collaboration and data sharing in a trusted, governed environment."

While organizations can collect data easily, it is the application of this data to a defined business strategy that is harder to implement, according to the research. A complete data intelligence strategy enables business users to master data access and governance while improving team collaboration which is essential for achieving value from data and analytic projects and enabling trust in the analytical outcomes. With only 16 percent of respondents stating that business users and analysts rate or comment on analytical outputs, data intelligent frameworks with stewardship and collaboration will be the competitive differentiator that improves efficiency, integrity and time to insight.