From the AI Native Dev: Intercom Co-Founder on AI Autonomy, Adapting Dev to LLMs and what AI Native means: Insights from Des Traynor

Introduction

In this episode of the AI Native Dev podcast, Guy Podjarny talks with Des Traynor, the co-founder of Intercom, about the company's journey in integrating AI into their customer support platform. Des, with his extensive background in product strategy and user experience, shares insights into the strategic decisions, challenges, and learnings from their shift to an AI-first approach. Before co-founding Intercom, Des worked as a User Experience Lead at Contrast and has been recognized for his influential thoughts on product design and customer engagement. His extensive experience in the software development industry and his role in pioneering the use of AI in customer support make him a trusted voice and authority in the field. This episode provides valuable lessons for any organization looking to leverage AI in their products and services.

The Evolution of Intercom’s AI Journey

Intercom started as a general-purpose customer communication tool, aiming to facilitate interactions between internet businesses and their customers. Des Traynor explained that about two years ago, Intercom shifted its focus to customer support. This transition was significantly influenced by the launch of ChatGPT, which prompted the company to adopt an AI-first approach. Des mentioned, "two years ago we went all in on customer support, and two, I guess maybe four weeks later, ChatGPT launched and then AI first customer support became the most obvious sort of future."

The introduction of AI-native features like Fin and Fin Copilot marked a pivotal moment in Intercom's strategy. These tools were designed to enhance customer support by leveraging AI to handle queries more efficiently and effectively. The decision to prioritize AI was immediate, with Des recalling, "By December 1st, we were working on what would be like the sort of future of our platform."

The Natural Fit of Chat-Based Support with AI

Intercom's existing chat-based support system naturally aligned with advancements in AI. Guy Podjarny noted, "I think of intercom oftentimes as the one that pioneered or at least popularized the, the sort of chat based support." The synergy between the rise of messaging and AI led to the development of effective chatbots, which matured significantly with the advent of generative AI.

Des pointed out that chatbots had multiple false starts before generative AI made them more effective. "Generative AI is the thing that made them like not shit, which is where the industry needed to get to," he said. The integration of AI transformed customer interactions, enabling always-on support and quick, accurate responses. Des added, "you could literally talk to the people behind a Tessl or a Snyk or whatever, just by engaging in the bottom right hand corner and asking them questions, get a quick answer."

The Code Red Moment and Rapid Adaptation

The critical moment for Intercom came when they recognized the potential of ChatGPT. Des described the timeline, "ChatGPT launches on a Thursday, it's Thursday evening in Dublin, Ireland, where we're playing with it. I'm slacking with our, VP Fergal of he's our VP of AI. We really, he's very convinced and he convinces me with examples that like, this is a thing, this is a big thing."

The swift decision-making process involved key figures like VP of AI Fergal and CEO Owen. By Monday, they had prioritized the first wave of AI features, focusing on augmenting support workflows. Des emphasized the importance of having a decisive CEO, "this is a time in a company where you need an extremely decisive and fast moving CEO who's not in any way, say, paralyzed by risk or fear."

Within seven weeks, Intercom released features like automatic summarization and reframing, which augmented the support workflow by removing repetitive tasks. The rapid development and deployment of these features showcased Intercom's ability to adapt quickly to new technological opportunities.

From AI-Assisted to Autonomous AI

Intercom's journey evolved from AI-assisted tools to autonomous AI agents. Initially, AI features focused on tasks like summarization and reframing for support reps. Des explained, "we were just trying to like, basically augment their workflow by removing all the undifferentiated heavy lifting."

The introduction of Fin marked the transition to autonomous AI. Fin could handle customer queries end-to-end, providing quick and accurate responses. The development of GPT-4 further enhanced Fin’s capabilities by reducing hallucinations. Des noted, "we realized, if this thing, if we can control the hallucinogenic properties of this, and dial them down, we can really build the end to end answer to damn question bot."

Challenges in Adapting Development Practices

Integrating AI into Intercom's platform required significant changes in software development practices. Des highlighted the need for enhanced data science skills and the ability to measure probabilistic outcomes. "To actually be a performant engineer in this era requires some amount of ability to interrogate data or perform data science, to measure outcomes," he said.

Continuous evaluation and A/B testing became crucial to ensure AI performance. Des mentioned that they ran around 50 different A/B tests on Fin’s performance to fine-tune its capabilities. Maintaining a balance between AI innovation and traditional development workflows was essential to ensure that new features did not negatively impact the existing system. "There's a very much you need to work out. You have to understand, trick to working with these systems is to like, bottle up a probabilistic outcome and productize it," Des explained.

Organizational Changes and Skill Set Evolution

The integration of AI also led to organizational changes and the evolution of skill sets within Intercom. Des distinguished between engineers in the AI group and those in the core product group. The AI group required data-driven decision-making and performance analysis skills. "The extra skills we look for within the AI group is everything around, data science. It's everything around basically like data driven decisions," Des said.

Hiring practices evolved to include more data science and machine learning expertise. Des noted, "The extra skills we look for within the AI group is everything around, data science. It's everything around basically like data driven decisions, and it's got there's a hard requirement on the ability to interrogate performance."

There was also an ongoing debate on the role of centralized vs. distributed AI capabilities within teams. Des mentioned that while the AI group was centralized, there were discussions about integrating AI capabilities more broadly across different teams.

The Future of AI-Powered Customer Support

Des shared his predictions for the future of AI in customer support. He envisioned a world where AI-driven interactions are both proactive and reactive. "It'll be both proactive and reactive. it'll, if I log in and my credit card's about to expire, it'll be like, yo, your credit card's about to expire. Click here to fix it or whatever," he said.

He emphasized the expectation of sub-second response times and improved support quality. "Humanity will have gotten used to and expect sub second replies to things," Des added. The evolving role of support professionals would involve managing AI policies and intelligence, ensuring that the AI systems align with the company's customer service philosophy.

Lessons Learned and Key Takeaways

Des highlighted several key lessons from Intercom's journey in integrating AI:

• Up-to-Date Knowledge Bases: Having accurate and up-to-date knowledge bases is crucial for AI performance. Outdated information can lead to incorrect responses and customer dissatisfaction.
• Disciplined Adoption: It's important to adopt AI tools cautiously and ensure they deliver the desired outcomes without negative impacts.
• Human Oversight: Maintaining a balance between human oversight and AI autonomy is essential to deliver quality customer support.
• Organizational Impact: AI integration affects organizational processes and requires a shift in development practices and skill sets.