A staggering 89% of respondents expressed concerns about providing private information to chatbots, with 11% stating they would never do this. While 50% of respondents have experienced biased responses and 38% have seen inaccuracies, another 75% believe chatbots are getting better at managing toxic or inaccurate responses.
The survey collected input from over 6,300 consumers, software developers and QA testers globally regarding their usage of, and attitudes toward, Gen AI tools and Gen AI in software testing. Despite the lingering concerns around data privacy, inaccurate and biased responses, Gen AI is now acknowledged for its practical applications. Research was found to be a popular use case and Gen AI is increasingly being used in software development and testing.
91% of respondents use chatbots to conduct research, with 33% stating they do so daily. It was also revealed that chatbots are replacing traditional search engines with 81% of respondents using chatbots to answer basic search queries, and 32% do so daily. However, only 19% of users indicated that the chatbot understood their prompt and provided a helpful response every time. Chatbot features users would like to see include better source attribution, more localized responses, support for more languages and deeper personalization.
Of the 1,539 respondents using Gen AI for software development and testing, the most common applications are writing or debugging code (51%), test reporting (48%), building test cases (46%) and building apps (42%). GitHub Copilot is the most popular tool for coding assistance (41%), followed by OpenAI Codex (24%).
Overall, it was no surprise to learn that ChatGPT was the most used chatbot (91%), followed by Gemini (63%) and Microsoft Copilot (55%). Other chatbots have been used by under a third of users: Grok (32%), Pi (29%), Perplexity (24%), Claude (23%) and Poe (21%).
The survey also found that it’s not unusual for users to switch between chatbots. 38% of respondents indicated that they use different chatbots depending on the specific task. 27% of respondents said they have replaced one chatbot with another due to performance. It was also revealed that multimodal capabilities are essential to getting value from chatbots. 62% said that multimedia is essential for a large portion of their usage of a Gen AI tool. A fourth of respondents have used both text and voice commands to interact with chatbots, with 5% stating they use voice as their main form of input.
“It’s clear from the survey that consumers are keen to use Gen AI chatbots, and some have even integrated it into their daily lives for tasks like research and search. Chatbots are getting better at dealing with toxicity, bias and inaccuracy – however, concerns still remain. Not surprisingly, switching between chatbots to accomplish different tasks is common, while multimodal capabilities are now table stakes,” said Chris Sheehan, SVP Strategic Accounts and AI at Applause. “To gain further adoption, chatbots need to continue to train models on quality data in specific domains and thoroughly test across a diverse user base to drive down toxicity and inaccuracy.”
