The importance of TDM

In today's economic climate, uncertainty is a common theme for businesses. With budget losses, high competition and job cuts being seen across several industries, avoiding unnecessary expenses is of the utmost importance. At the same time, though, organisations are still keen to be seen as successful, productive and agile. By Pooja Tyagi, Testing Engineering Specialist at NTT DATA UK.

Monday, 22nd April 2019 Posted 5 years ago in by Phil Alsop

Every business needs to be innovative and tech-savvy. It’s no longer the case that software implementation and development is solely the IT department’s problem. Increasingly, business elements which would have previously been considered purely a ‘tech issue’ now hold the key to advancing in the market or solving a wider business issue. Ignoring or misunderstanding the importance of such elements can therefore have serious implications for the business as a whole.

One of these previously niche concerns that is now impacting wider business efficiency is test data management (TDM), which facilitates test data during various phases of a software development life cycle. A critical component in the testing stage of developing software and apps, TDM efficiently manages and streamlines data to help businesses save time, efforts and costs. As the popularity of TDM continues, what trends are driving this growth and what are the challenges this practice encounters?

Trends and accelerators

Domain knowledge is key to determine efficient approaches for identifying data subset criteria and masking methods, as well as understanding data creation techniques to ensure data integrity. In fact, many service providers are now offering joint TDM solutions along with tool vendors. But what is driving this increase?

Currently, testers have to perform integration tests on applications using different code versions and test data combinations. With other applications (in-house or vendor) these methods are not currently available. Due to this, test data architects need solutions to virtualise application behaviour with predefined test data combinations (input and output) and orchestrate it with the application under test. This is where TDM can really shine.

While potentially considered “buzzwords”, some key trends and solution patterns that must be considered while creating effective test data include DevOps, big data, cloud and service virtualisation with TDM. Other common themes being seen at the moment include domain-specific solutions and automation, which can be used to reduce time-to-market and improve efficiency. These trends are very visible in product vendors’ roadmaps and would always be a main expectation from the IT organisation during their TDM journey.

Best Practices

More often than not, the missing piece of the software testing puzzle is data virtualisation. Unlike physical data, "virtual data" can be rapidly provisioned and accurately simulated in minutes, making it easy to distribute test data to application teams. And unlike storage replication tools, data virtualisation also equips end users with self-service data controls, meaning software testing can be performed iteratively without unnecessary delays.

The traditional request-fulfil model takes days or weeks with multiple handoffs to complete. Conversely, self-service data is accessible in minutes, meaning administrators can quickly deliver this virtual data to application teams and execute repeatable masking algorithms to streamline the handoff from Ops to Dev in a more efficient manner.

The challenges to TDM

As a practice, TDM is currently being impacted by three major trends: the digital revolution, an increase in the adoption of data analytics, and pressure from the business to deliver better software, faster.

Most notably, creating a test environment with the appropriate test data is a slow, manual, and high-touch process. Many organisations still use a request-fulfil model, in which ticket requests are often delayed or unfulfilled. Copying a dataset can take days or weeks and involve multiple handoffs between teams, creating a fundamental process bottleneck.

In terms of specific challenges to the practice, there is a lack of awareness and standardisation to TDM, as well as restrictions caused by regulatory compliances. However, aspects such as high storage cost and poor data quality, as well as an absence of traceability are what can, traditionally, give TDM a bad name.

Is TDM for you?

So, having weighed up the pros and the cons, it’s clear that saying ‘yes’ or ‘no’ to Test Data Management offers contrasting outcomes. Of course, the underlying objective is to deliver value to clients by increasing automation through efficient solution patterns and innovative methods across the TDM lifecycle.

With newer technologies and approaches comes changes in techniques of testing, as well as the data sufficient to conduct these tests. This means that today, test data management is more relevant to businesses than ever.