New Relic customers will now be able to monitor their custom ChatGPT apps to ensure they’re delivering the intended performance, reliability, and user experience.
“Bringing business services into the natural flow of a ChatGPT conversation is a powerful, intuitive, and revenue-generating strategy,” said Brian Emerson, chief product officer of New Relic. “But once your carefully crafted application instantiates inside ChatGPT, it traditionally enters a black box where standard browser monitoring tools can fail.”
The company went on to explain that when an app is rendered in a conversation, developers can’t see things like layout shifts or broken buttons. Additionally, security headers, content security policies, i-frame sandbox rules, and limitations on client-side storage can hide important performance and user experience data.
New Relic’s answer to this problem is to send in an agent that can collect and analyze data. It can track PageViews, PageViewtimings, and AjaxRequests, providing insights into latency and connectivity within the GPT i-frame. It will alert developers if an AI response triggers script or syntax failures, and also logs those events to the console.
It additionally tracks how users interact with content, such as not engaging at all or clicking the “buy now” button. Developers will be able to build dashboards that track AI-specific benchmarks, like “AI Render Success Rate,” which tracks how often the LLM successfully populates a chart according to the specification, and “Prompt-to-Action Conversion,” which tracks if a specific type of suggestion leads to more user engagement.
“With New Relic, developers can stop guessing how their app performs when it’s hosted by someone else, while also maintaining the highest security and privacy standards,” the company wrote in an announcement.
