From the AI Native Dev: Enterprise AI Solutions Need to be Different - Glean and ex-Slack CPO, Tamar Yehoshua, on RAG, Changing Behavior and Bring-Your-Own-Model.

Introduction

In this episode of AI Native Dev, host Guy Podjarny interviews Tamar Yehoshua, a seasoned tech leader with an illustrious career in engineering and product leadership roles at companies like Amazon, Google, Slack, and Glean. Tamar currently serves as the President of Products and Technology at Glean. The conversation delves into the intricacies of AI in the enterprise, building AI-based products, and the challenges of integrating AI solutions in a secure and effective manner.

Tamar Yehoshua's Career Journey

Tamar Yehoshua has had an impressive career trajectory marked by significant roles at several top-tier tech companies. She has held leadership positions at Amazon, where she was a VP at A9, and at Google, where she played a pivotal role in Search. Her journey also includes being the Chief Product Officer at Slack, where she met Guy Podjarny during her time as a board member at Snyk.

Reflecting on her career, Tamar shared, "The most important part was being on the board at Snyk where I got to meet you." Her current role as the President of Products and Technology at Glean involves overseeing the development and integration of AI solutions within the enterprise context.

Introduction to Glean

Glean is an enterprise AI platform designed to ingest information from various SaaS applications and make it accessible through natural language queries. As Tamar explained, "Glean is an enterprise AI platform. What we do is we read the content of all of your SaaS applications... and make it easy to find information across all of your SaaS tools."

Glean's capabilities extend beyond simple search functions. It creates a knowledge graph of enterprise data and offers a platform for building no-code applications. This allows users to query their enterprise data and receive summarized answers, much like a ChatGPT for organizational knowledge.

The Dual Nature of Glean: Search and Chat

Glean offers two primary interfaces: a Google-like search interface and a ChatGPT-like interface. This duality enables users to either search for specific information or generate summarized answers based on their queries. Tamar illustrated this with an example: "You can use it for both. So let's say I'm going on a customer call and I want to know who's the account executive for it. I can ask Glean really simply."

The search interface is ideal for discovering specific documents or pieces of information scattered across various enterprise systems. In contrast, the chat interface allows for more conversational queries, making it easier to get comprehensive answers quickly.

Security and Privacy in AI Solutions

Security and privacy are paramount in any enterprise AI solution. Tamar emphasized that Glean takes these concerns seriously: "We have over a hundred connectors to all your different SaaS apps and we understand the privacy and governance of each app." Glean adheres to strict data privacy policies and ensures that only authorized personnel can access specific data.

To further enhance security, Glean can be hosted in a customer's own cloud environment, such as GCP or AWS. This ensures that sensitive data does not leave the company's premises, providing an additional layer of security.

Architectural Insights: RAG-based Solution

Glean's architecture is built on a Retrieval-Augmented Generation (RAG) based solution. This approach ensures that the system retrieves relevant documents based on user queries and then uses an LLM to generate comprehensive answers. Tamar explained the process: "We call the LLM for the planning phase, where we take that query and we translate it into search queries. Then we create our retrieval engine and we get back the relevant snippets."

This architecture allows Glean to maintain high accuracy and security, as the LLM only processes data that the user is authorized to access.

Evolution from AI to Gen AI at Glean

Glean started as an AI search company in 2019, using AI techniques to improve enterprise search capabilities. Tamar recounted the journey: "Glean started in 2019... as building enterprise search. But because he [Arvind Jain] came from Google, he knew that AI was being used in search at the time."

The transition to incorporating generative AI capabilities was a natural expansion. With the advent of GPT-3, Glean quickly integrated these new technologies to enhance its product offering. Tamar noted, "There was a war room, get everyone together and build the assistant."

Challenges in Building AI Solutions for Enterprises

Building AI solutions for enterprises comes with unique challenges, including handling non-deterministic results and managing user expectations. Tamar highlighted the importance of educating users about what AI can and cannot do: "The biggest stumbling block we have is people understanding what they can and can't do with it."

To evaluate and improve its AI models, Glean employs several methods, including using LLM as a judge and LLM jury. These techniques help ensure that the AI provides accurate and reliable answers, even in complex enterprise environments.

User Adoption and Behavior Change

Changing user behavior and educating them about AI capabilities is crucial for successful adoption. Glean employs strategies like prompt suggestions and Glean Apps to guide users. Tamar mentioned, "One of the things about Glean is that we understand your organization... So we can suggest prompts that people in your team have been using."

These strategies help users quickly understand how to leverage AI tools effectively, making the transition smoother and more intuitive.

The Future of AI in the Enterprise

Looking ahead, Tamar is optimistic about the future of AI in the enterprise. She envisions a world where AI assistants handle repetitive tasks, allowing employees to focus on more creative and high-leverage activities. She shared, "People are going to be able to automate away a lot of what they do today, a lot of the toil that they do today."

This shift will not only increase productivity but also enhance job satisfaction by enabling employees to spend more time on meaningful work.

Summary

In this insightful conversation, Tamar Yehoshua shared her career journey and the innovative work being done at Glean. Key takeaways include:

• Glean's dual interface approach for search and chat.
• Emphasis on security and privacy in AI solutions.
• The evolution from AI to generative AI.
• Challenges and strategies in building and adopting AI solutions.
• The promising future of AI in automating tasks and enhancing productivity.

For organizations looking to implement AI solutions, these insights provide valuable guidance on navigating the complexities and opportunities that AI presents.