Slow Data Kills Business

More and more businesses today are looking to extract insight from the data they have access to in as near real-time as possible. They are looking to rapidly gain the insight and intelligence they need to drive optimum customer experiences; faster time to value and a competitive edge. By Jeff Fried, Director, Product Management - Data Platforms, InterSystems.

Monday, 10th December 2018 Posted 6 years ago in by Phil Alsop

These are elusive goals for many businesses though. Companies need to be able to blend analytical queries and transactional data. Otherwise, they are likely to be basing decisions on data that is anywhere from ten minutes to two hours or even days old, making it all but impossible to capitalise on many real-time and near real-time business opportunities.

It is hardly surprising, then, that, according to Choosing a DBMS to Address the Challenges of the Third Platform, May 2017, an IDC InfoBrief sponsored by InterSystems, 76 per cent of respondents reported that the inability to analyse current data inhibits their ability to take advantage of business opportunities.¹

Bringing Together the Transactional and the Analytic

In assessing how a combination of transactional and analytic data processing can help here, it makes sense to first consider their respective roles within the enterprise. One of their distinguishing features today is simply that they are separate. Indeed, transactions and analytics typically form two distinct data processing arms within the enterprise.

Transactions often involve the processing of records data in relation to regular operations conducted across the entire business and are designed for write, not query, speed. Analytics process data from multiple transactional databases and are designed for query speed to provide organisations with insights based on specific questions.

Data often needs to move from transactional systems to analytics, increasing complexity and latency that slows the business down and can lead to missed opportunities. Transactional data processing is often limited in its ability to quickly perform analytic queries, while analytics data processing is often too slow to deliver valuable real-time insights. A transactional approach drives business operations. Analytics make the data actionable and bring out its value, empowering organisations to identify connections across multiple transactional databases.

According to the IDC study, 86.5 percent of organisations use ETL to move at least 25 percent of all enterprise data between transactional and analytical systems. And nearly two-thirds (63.9 percent) of data moved via ETL is at least five days old by the time it reaches an analytics database.² This is a

critical obstacle for most organisations that want to deliver the right customer experience at the right moment.

Businesses do, however, also face other hurdles on top of this. Typically, they will need to support more data types (structured, unstructured, etc.), larger data sets and an accelerated path from analysis to action introduced by mobile users, IoT/sensor data, and fickle / constantly emerging trends.

This situation is not helped either by the disparate range of data management tools they typically use today. Companies often utilise several different database systems, for example, which means the data is saved and stored in a wide range of different places and formats. Each database is unique to the types of data and types of workload it specifically manages.

More than 60 percent of respondents to the recent IDC survey reported having more than five

analytical databases, and more than 30 percent have more than 10.³The majority of respondents have more than five production transactional databases, while 25 percent had more than 10.⁴

Companies now have the challenge of harnessing that data and determining how to extract value from it by applying it to business operations – making sense of all the data by tying all the sources to an individual customer, patient, citizen, investor, etc.

Finding a Solution: What is a Modern Data Platform?

By combining analytic and transactional data processing, including a range of data types in support of digital transformation, a modern data platform lies at the heart of a data-driven business. A data management platform is a centralised computing system for collecting, integrating, managing and analysing large sets of structured and unstructured data from disparate sources at massive scale (distributed as well as single server) and can support multiple use case scenarios and workloads (transaction processing and analytics) with native data and application interoperability.

There are three central pillars to a modern data platform.

It must support all data types and data processing (analytics and transactions)
It must be scalable, flexible and interoperable
It must be reliable, high performance and have zero latency

Why do companies need a consolidated data platform? Because disparate systems create a disconnect between insight and action, resulting in a delay in the feedback loop that drives the ultimate customer experience. A consolidated data platform helps companies achieve their core (IT-related) business objectives, such as simplifying architecture, reducing cost, speeding innovation and streamlining operations.

Connecting Insight and Action

Managing multiple databases is complex, expensive and introduces latency issues. As data itself grows more complex, deploying a unique database and data integration system for each business need creates unnecessary complexity. It means tactical decisions cannot be supported as long as data is segregated into transactional and analytical databases. Most users need a broad variety of data type support that goes well beyond what native relational database management systems (RDBMS) provide. Lastly, the rising costs of database management are cost prohibitive to many organisations. The maintenance of many databases leads to excessive cost and complexity in the data centre.

Delivering Ultimate Data-driven Experiences

How do you define the ultimate experience for your customers, partners and stakeholders? What data do you need to ensure all the information can be accessed for both guiding a decision and supporting and executing a decision (transaction)?

To answer these questions, you must first identify your organisation’s data infrastructure needs. This includes understanding where your data resides, how often and by what means it is accessed, and how it is being analysed.

Companies no longer need to choose between having real-time access to data and their preferred method of analysing data. They can access data how and when they want and have the ability to make the data actionable in real-time. Modern data platforms help make your data infrastructure work for your business.

Timely access to data can have a major impact on company operations and customer experience. As more and more data is generated by companies and their customers, it is important to be able to easily access and analyse this information and use it to inform business decisions. A modern data platform simultaneously supports analytical and transactional decisions and streamlines data infrastructure costs, driving more intelligent insights across the entire organisation.