Dynatrace launches AI Observability for Large Language Models and Generative AI

Dynatrace has extended its analytics and automation platform to provide holistic observability and security for large language models (LLMs) and generative AI-powered applications. This enhancement to the Dynatrace® platform enables organizations worldwide to embrace generative AI confidently and cost-effectively as part of their focus on increasing innovation, productivity, and revenue.

Dynatrace® AI Observability is a comprehensive solution. It covers the end-to-end AI stack, including infrastructure, such as Nvidia® GPUs, foundational models, such as GPT4, semantic caches and vector databases, such as Weaviate, and orchestration frameworks, such as LangChain. It also supports the major platforms for building, training, and delivering AI models, including Microsoft® Azure OpenAI Service, Amazon® SageMaker, and Google® AI Platform.

Dynatrace AI Observability leverages the platform’s Davis® AI and other core technologies to deliver a precise and complete view of AI-powered applications. As a result, organizations can provide great user experiences while identifying performance bottlenecks and root causes automatically. Dynatrace AI Observability with Davis AI also helps them comply with privacy and security regulations and governance standards by tracing the origins of the output created by their apps with precision. Additionally, it helps them forecast and control costs by monitoring their consumption of tokens, which are the basic units that generative AI models use to process queries.

“Generative and predictive AI will unlock new possibilities for our business with our ML and LLM services, but to implement them successfully, we need to ensure that our services supporting these critical workloads are reliable and perform well,” said Ryan Berry, SVP Engineering & Architecture at OneStream. “That’s why we rely on Dynatrace, a leader in AI and observability. Our teams use Dynatrace to build and optimize generative AI apps that perform well and are cost effective to manage and deploy at scale.”

Gartner® states, “By 2028, the adoption of AI will culminate in over 50% of cloud compute resources devoted to AI workloads, up from less than 10% in 2023.[i]” This growth partially reflects organizations’ surging interest in generative AI to improve efficiency and productivity, drive automation, and foster innovation. Despite this momentum, many organizations are concerned about costs associated with generative AI-powered services. These can be many times more expensive than traditional cloud services and are difficult to forecast because they are based on the consumption of generative AI tokens by applications that are not yet in production. Simultaneously, governments worldwide are establishing regulations focused on the use of AI technologies in a responsible and ethical manner and in compliance with applicable laws.

Organizations cannot afford to ignore the potential of generative AI. However, they need comprehensive AI observability to help their generative AI investments succeed and avoid the risk of unpredictable behaviors, AI “hallucinations,” and bad user experiences.

“Generative AI is the new frontier of digital transformation,” said Bernd Greifeneder, CTO at Dynatrace. “This technology enables organizations to create innovative solutions that boost productivity, profitability, and competitiveness. While transformational, it also poses new challenges for security, transparency, reliability, experience, and cost management. Organizations need AI observability that covers every aspect of their generative AI solutions to overcome these challenges. Dynatrace is extending its observability and AI leadership to meet this need, helping customers to embrace AI confidently and securely with unparalleled insights into their generative AI-driven applications.”

“AI-powered transformation is a key driver of productivity and competitiveness for organizations worldwide,” said Ali Dalloul, VP, AI, Microsoft. “Microsoft Azure OpenAI Service is a robust generative AI solution for enterprises and public sector entities across the globe. We're pleased to partner with Dynatrace, a leader in AI-powered observability, to empower our shared customers, the teams driving innovation in the world’s largest organizations, to create generative AI-based services using Azure Open AI Service and Dynatrace AI Observability. This powerful combination helps ensure these services achieve the highest security, reliability, and performance standards while enabling the teams that manage them to control costs.”