Data warehousing dissatisfaction

New research published by SnapLogic reveals that 83% of organizations are not fully satisfied with the performance and output of their data management and data warehousing initiatives. IT leaders cite a growing number of disconnected applications and data sources, outdated legacy systems, and slow and manual data movement as reasons for their frustration, all of which are stalling progress and costing them millions.

The new study, conducted by independent research firm Vanson Bourne, found that the average organization has 115 distinct applications and data sources across their enterprise, but almost half of them (49%) are siloed and disconnected from one another. Respondents expressed clear concern, with 89% of IT Decision Makers (ITDMs) from organizations where these apps and systems aren’t integrated worried that these data silos are holding them back. Indeed, ITDMs confirmed they are losing, on average, more than $1 million USD annually due to poor data management.

Still, organizations are committed to building a data-driven enterprise and are funding their data management initiatives accordingly. Over three quarters (76%) of survey respondents indicated that their companies have increased their data budgets over the past year.

Delivering on the Promise
ITDMs are investing in data storage solutions such as data warehouses and data lakes to help them make the most of their valuable data assets and deliver on the promise of agile analytics and actionable business insights. With this data, respondents hope to achieve faster and more informed decision making (70%), an improvement in business operations (64%), accelerated innovation and new business development (63%), an enhanced customer experience (61%), and more efficient and productive employees (57%). However, the process of getting data into a data warehouse or data lake is not always straightforward, all too often inhibiting progress and success.

Key research findings:

●      Nearly nine in ten (88%) ITDMs experience challenges trying to load data into data warehouses, with the biggest inhibitors being legacy technology (49%), complex data types and formats (44%), data silos (40%), and data access issues tied to regulatory requirements (40%).

●      Nearly half (48%) of the data migrated into organizations’ data warehouses or other storage solutions requires cleaning before it can be useful. This likely contributes to the more than 4 hours of time wasted on average, per employee per week, resolving issues around missing data, duplicate data, or data that needs to be reformatted.

●      ITDMs indicated that, on average, 42% of their data management and warehousing processes are currently being done manually, but could be automated, which would save valuable time, resources, and money.

●      As a result of these issues, almost all respondents (93%) believe improvements are needed in how they collect, manage, and analyze data.

“We are not only in a period of exponential data growth, but also one in which data lies at the heart of the successful adoption of technologies such as AI, automation, and advanced analytics,” said Craig Stewart, CTO at SnapLogic. “This means that having an effective data management and warehousing strategy is business-critical”.