New Relic is introducing a new observability solution aimed at applications running within ChatGPT. With this expansion, the company is responding to a growing problem for organizations that use generative AI as a distribution channel: the lack of insight into what happens once an application is run in an AI-driven environment.
More and more companies are developing custom applications that are accessible directly from ChatGPT. In doing so, they are moving parts of their services to OpenAI’s infrastructure. This increases their reach, but also makes it more difficult to gain insight into performance and usage. Traditional browser monitoring has limited functionality within the protected context in which ChatGPT apps are executed.
According to New Relic, applications often disappear into a technical blind spot after integration. Developers cannot see how their software is performing, where errors occur, or how users are behaving, even though this information is precisely what is needed to improve conversions and identify problems at an early stage. The new solution aims to restore this insight by making performance, reliability, and user experience visible, even when an application is running within an i-frame or sandbox environment.
A major bottleneck with ChatGPT apps is that many common signals are missing. Layout problems, non-functioning buttons, or why users terminate the interaction prematurely are difficult to identify when an application does not have control over the entire browser environment. This is exacerbated by the nature of generative AI, where dynamically generated interfaces look correct but do not function properly from a technical standpoint. The AI may also refer to data that was never provided by the backend, without this being immediately visible to development teams.
Browser telemetry applied to ChatGPT context
New Relic addresses this problem by applying existing browser telemetry within the ChatGPT context. The company’s browser agent collects data on PageViews, PageViewTimings, AjaxRequests, and more. This provides insight into latency and connectivity within the GPT-i frame. In addition, the agent flags when AI-generated output leads to script or syntax errors on the client side. It also records these events via console logs. According to New Relic, this provides real-time visibility into the actual user experience, even in an environment that normally offers little transparency.
In addition to performance data, the platform provides insight into how users interact with the content of a ChatGPT app. Development teams can record which interactions are important to them and link them to measurement points. This makes it possible to relate the functioning of AI-generated content to user behavior, such as continuing or dropping out, without relying on assumptions.
The solution is designed for end-to-end visibility. Interactions that take place within ChatGPT can be tracked back to backend services, making it clear how user actions affect underlying systems. According to New Relic, this is done within the platform’s existing security and privacy frameworks, despite the limitations that apply within i-frames and sandbox environments.
Monitoring for ChatGPT apps is part of New Relic’s Intelligent Observability Platform and is available immediately. Users must install the latest browser agent and determine for themselves which interactions are critical. With this, New Relic is focusing explicitly on observability for AI-driven applications, an area that is becoming increasingly relevant as generative AI is used more often as an interface for business services.