Continued growth in AI and automation for data collection

The web intelligence industry has decisively turned to artificial intelligence as the main method to improve its operations.

  • Monday, 20th January 2025 Posted 4 months ago in by Phil Alsop

77% of web intelligence professionals have tried using AI-powered web scraping tools, with 86% stating that it saves time and resources. This is according to new research from leading web intelligence acquisition provider Oxylabs.

In cooperation with Censuswide, Oxylabs surveyed 506 software engineers and web scraping professionals from UK and US companies, gathering insights on the issues faced when collecting web intelligence and the impact it brings to businesses.

Findings from the research confirm that with advances in AI model development, the demand for high-quality public web data is accelerating leading into 2025. 47% of respondents claim to use AI-powered solutions regularly, while 20% have not used them so far but are keen to do so, demonstrating that the web intelligence industry has embraced AI adoption and the benefits it will bring.

Juras Juršėnas, Chief Operations Officer at Oxylabs, said, “Many industries have been hesitant to adopt AI tools, doubting the impact they provide and being unwilling to risk the investment. Yet, AI tools are perfectly suited for larger-scale web data collection. Building and maintaining data pipelines involves lots of moving cogs and an abundance of repetitive tasks that AI can streamline.”

67% of respondents stated that AI will primarily help build data parsers across the web intelligence industry, followed by target unblocking at 59%. Nevertheless, 43% of respondents saw target unblocking as the most suitable for AI applications, as it is a more sophisticated task and a more fitting challenge for the newest technological developments, such as AI tools.

Julius Černiauskas, Chief Executive Officer at Oxylabs, said, “Parsing has remained time-consuming and challenging even when AI was applied, as there have been few exceptional AI-based solutions specialised in data parsing on the market up to now. However, new AI tools such as Oxylabs OxyCopilot are emerging to assist with this. These tools include custom parser builders which enable data parsers to be created in minutes, and request builders to help define scraping requests with only minimal parameters.”

The combination of these two functionalities will help new AI tools to save developers’ time, avoid business costs created by heavily delayed data collection, simplify onboarding for those with less web scraping experience, and reduce spending on internal computing resources.

“The web intelligence industry is a long way from full AI integration, but new innovations are making a major difference in streamlining many tasks. In 2025, web intelligence companies should closely watch how these trends are developing and take steps to leveraging new tools at every opportunity”, concluded Černiauskas.

Technical debt stifling path to AI adoption

Posted 1 day ago by Phil Alsop
Outdated legacy technologies costing organizations the ability to innovate, money, time – and, potentially, even customers.
Data from ‘Unlocking Growth in the Mid-Market: The Node4 Report’, reveals UK mid-market leaders are taking a more pragmatic approach to public...
According to research unveiled today, one in five CIOs and CTOs at enterprise companies (21%) believe that their organisations’ road to digital...
76% of financial services firms surveyed plan to implement agentic AI within the next year.

Why most businesses aren’t yet winning with AI

Posted 4 days ago by Phil Alsop
71% of business leaders say their workforces are not ready to successfully leverage AI.
Five9 has released its 2025 Business Leaders Customer Experience Report offering analysis of CX trends shaping how global business leaders create...
Delinea has unveiled new research highlighting how ransomware attacks have continued to surge over the past year, despite fewer victims paying. Over...
96% of tech professionals view AI agents as a growing security risk, yet 98% of organisations plan to expand adoption.