CtrlK

Community Documentation Log in Get started

tessl/maven-com-embabel-agent--embabel-agent-rag-core

RAG (Retrieval-Augmented Generation) framework for the Embabel Agent platform providing content ingestion, chunking, hierarchical navigation, and semantic search capabilities

Overview

Eval results

Files

Quickstart: LLM Integration

Name: tessl/maven-com-embabel-agent--embabel-agent-rag-core
Author: tessl

This guide covers integrating RAG capabilities with Large Language Models using ToolishRag, including tool configuration, filtering, monitoring, and Spring AI integration.

Overview

LLM integration enables language models to:

Search documentation - Retrieve relevant content using vector or text search
Find entities - Look up people, projects, or domain objects
Expand context - Get surrounding chunks or parent sections
Use filters - Apply metadata and entity filters automatically

Basic ToolishRag Setup

Create a simple RAG tool for LLM access.

import com.embabel.agent.rag.tools.ToolishRag
import com.embabel.agent.rag.service.*

val searchOperations: CoreSearchOperations = // implementation

// Create basic RAG tool
val ragTool = ToolishRag(
    name = "documentation_search",
    description = "Search technical documentation and user guides",
    searchOperations = searchOperations
)

// Get tools for LLM
val tools = ragTool.tools()
println("Exposed ${tools.size} tools to LLM")

// Get usage notes for LLM context
val notes = ragTool.notes()
println(notes)

Understanding Generated Tools

// ToolishRag automatically creates these tools:
// - vectorSearch(query, topK, threshold) - Semantic similarity search
// - textSearch(query, topK, threshold) - Full-text search with Lucene syntax
// - broadenChunk(chunkId, chunksToAdd) - Get surrounding chunks
// - zoomOut(id) - Get parent section
// - findById(id, typeName) - Look up by identifier

tools.forEach { tool ->
    println("Tool: ${tool.name}")
    println("Description: ${tool.description}")
    println()
}

Configure Search Types

Specify which content types to search.

import com.embabel.agent.rag.model.*

val ragTool = ToolishRag(
    name = "knowledge_base",
    description = "Search knowledge base including docs, facts, and entities",
    searchOperations = searchOperations
)

// Configure search types
val configured = ragTool.withSearchFor(
    vectorSearchFor = listOf(
        Chunk::class.java,
        Fact::class.java,
        NamedEntity::class.java
    ),
    textSearchFor = listOf(
        Chunk::class.java,
        Fact::class.java
    )
)

Type-Specific Tools

// Create separate tools for different content types

// Documentation search
val docsTool = ToolishRag(
    name = "docs_search",
    description = "Search technical documentation",
    searchOperations = docsRepository
).withSearchFor(
    vectorSearchFor = listOf(Chunk::class.java),
    textSearchFor = listOf(Chunk::class.java)
)

// Entity search
val entityTool = ToolishRag(
    name = "entity_search",
    description = "Search people, projects, and organizations",
    searchOperations = entityRepository
).withSearchFor(
    vectorSearchFor = listOf(NamedEntity::class.java),
    textSearchFor = listOf(NamedEntity::class.java)
)

// Combine tools
val allTools = docsTool.tools() + entityTool.tools()

Apply Filters

Add default filters to all searches.

import com.embabel.agent.rag.filter.*

// Add metadata filter
val filteredRag = ToolishRag(
    name = "security_docs",
    description = "Search security-related documentation",
    searchOperations = searchOperations
).withMetadataFilter(
    PropertyFilter.eq("category", "security")
        .and(PropertyFilter.gte("version", 3.0))
        .and(PropertyFilter.ne("status", "archived"))
)

// All searches will automatically apply these filters

Entity Filters

// Filter by entity labels
val peopleRag = ToolishRag(
    name = "people_search",
    description = "Search employees and contractors",
    searchOperations = entityRepository
).withEntityFilter(
    EntityFilter.hasAnyLabel("Employee", "Contractor")
).withMetadataFilter(
    PropertyFilter.eq("status", "active")
)

Combined Filters

// Combine multiple filter types
val comprehensiveRag = ToolishRag(
    name = "platform_knowledge",
    description = "Search platform team knowledge base",
    searchOperations = searchOperations
)
    .withMetadataFilter(
        PropertyFilter.eq("team", "platform")
            .and(PropertyFilter.gte("lastUpdated", recentTimestamp))
    )
    .withEntityFilter(
        EntityFilter.hasAnyLabel("Chunk", "Fact", "Person")
    )

Add Search Hints

Use HyDE (Hypothetical Document Embeddings) to improve search quality.

import com.embabel.agent.rag.tools.TryHyDE

// Add HyDE hint using conversation context
val hydeRag = ToolishRag(
    name = "code_search",
    description = "Search code documentation and examples",
    searchOperations = searchOperations
).withHint(
    TryHyDE.usingConversationContext()
        .withMaxWords(75)
)

Custom Context HyDE

// Provide specific context for HyDE
val contextualRag = ragTool.withHint(
    TryHyDE.withContext(
        "Kotlin programming language best practices and design patterns"
    ).withMaxWords(60)
)

Multiple Hints

// Chain multiple hints (though typically you'd use one HyDE)
val multiHintRag = ragTool
    .withHint(TryHyDE.usingConversationContext())
    .withGoal(
        "Use this tool to find relevant information. " +
        "For code examples, try exact text search. " +
        "For concepts, use vector search."
    )

Custom Result Formatting

Format search results for optimal LLM consumption.

import com.embabel.agent.rag.service.*

// Create custom formatter
val markdownFormatter = RetrievableResultsFormatter { results ->
    buildString {
        appendLine("# Search Results")
        appendLine()
        appendLine("Found ${results.results.size} relevant matches:")
        appendLine()

        results.results.forEachIndexed { index, result ->
            appendLine("## ${index + 1}. Relevance: ${"%.1f".format(result.score * 100)}%")
            appendLine()

            when (val content = result.content) {
                is Chunk -> {
                    val section = content.metadata["container_section_title"]
                    if (section != null) {
                        appendLine("**Source:** $section")
                    }
                    appendLine()
                    appendLine(content.text)
                }
                is NamedEntity -> {
                    appendLine("**Entity:** ${content.name}")
                    appendLine()
                    appendLine(content.description)
                }
            }
            appendLine()
            appendLine("---")
            appendLine()
        }
    }
}

// Use custom formatter
val ragTool = ToolishRag(
    name = "knowledge_search",
    description = "Search knowledge base",
    searchOperations = searchOperations,
    formatter = markdownFormatter
)

Compact Formatter

// Minimal formatter for token efficiency
val compactFormatter = RetrievableResultsFormatter { results ->
    results.results.joinToString("\n\n") { result ->
        when (val content = result.content) {
            is Chunk -> "[${"%.2f".format(result.score)}] ${content.text}"
            is NamedEntity -> "[${"%.2f".format(result.score)}] ${content.name}: ${content.description}"
            else -> result.content.infoString()
        }
    }
}

Monitor Search Events

Track and analyze search operations.

import com.embabel.agent.rag.tools.*

// Create results listener
val listener = ResultsListener { event ->
    println("Query: ${event.query}")
    println("Results: ${event.results.size}")
    println("Time: ${event.runningTime.toMillis()}ms")
    println("Timestamp: ${event.timestamp}")

    // Log top result
    event.results.firstOrNull()?.let { top ->
        println("Top score: ${"%.3f".format(top.score)}")
    }
}

// Add listener
val monitoredRag = ToolishRag(
    name = "monitored_search",
    description = "Search with monitoring",
    searchOperations = searchOperations
).withListener(listener)

Analytics Listener

import java.util.concurrent.ConcurrentHashMap
import java.util.concurrent.atomic.AtomicLong

class SearchAnalytics {
    private val queryCount = ConcurrentHashMap<String, AtomicLong>()
    private val avgScores = ConcurrentHashMap<String, Double>()
    private val totalSearches = AtomicLong(0)
    private val totalTime = AtomicLong(0)

    val listener = ResultsListener { event ->
        // Track query frequency
        queryCount.computeIfAbsent(event.query) {
            AtomicLong(0)
        }.incrementAndGet()

        // Track performance
        totalSearches.incrementAndGet()
        totalTime.addAndGet(event.runningTime.toMillis())

        // Track quality
        if (event.results.isNotEmpty()) {
            val avgScore = event.results.map { it.score }.average()
            avgScores[event.query] = avgScore
        }
    }

    fun getStats(): Map<String, Any> {
        val searches = totalSearches.get()
        return mapOf(
            "totalSearches" to searches,
            "uniqueQueries" to queryCount.size,
            "avgLatency" to if (searches > 0) totalTime.get() / searches else 0,
            "topQueries" to queryCount.entries
                .sortedByDescending { it.value.get() }
                .take(10)
                .associate { it.key to it.value.get() },
            "avgScores" to avgScores.toMap()
        )
    }

    fun reset() {
        queryCount.clear()
        avgScores.clear()
        totalSearches.set(0)
        totalTime.set(0)
    }
}

// Use analytics
val analytics = SearchAnalytics()

val ragTool = ToolishRag(
    name = "analytics_search",
    description = "Search with analytics",
    searchOperations = searchOperations
).withListener(analytics.listener)

// Later, check stats
val stats = analytics.getStats()
println("Search analytics: $stats")

Customize Tool Behavior

Custom Goal

val ragTool = ToolishRag(
    name = "api_docs",
    description = "Search API documentation",
    searchOperations = searchOperations
).withGoal(
    "Use this tool to find API endpoints, request/response formats, and usage examples. " +
    "For exact endpoint names, use text search. " +
    "For conceptual queries, use vector search. " +
    "If results are unclear, try expanding chunks for more context."
)

Complete Configuration

import com.embabel.agent.rag.filter.*
import com.embabel.agent.rag.tools.*
import com.embabel.agent.rag.model.*

val comprehensiveRag = ToolishRag(
    name = "platform_knowledge",
    description = "Comprehensive platform team knowledge base",
    searchOperations = searchOperations
)
    // Configure search types
    .withSearchFor(
        vectorSearchFor = listOf(
            Chunk::class.java,
            Fact::class.java,
            NamedEntity::class.java
        ),
        textSearchFor = listOf(
            Chunk::class.java,
            Fact::class.java
        )
    )
    // Apply filters
    .withMetadataFilter(
        PropertyFilter.ne("status", "archived")
            .and(PropertyFilter.eq("team", "platform"))
    )
    .withEntityFilter(
        EntityFilter.hasAnyLabel("Chunk", "Fact", "Person", "Project")
    )
    // Add search hints
    .withHint(
        TryHyDE.usingConversationContext()
            .withMaxWords(60)
    )
    // Add monitoring
    .withListener { event ->
        println("Search: ${event.query} → ${event.results.size} results in ${event.runningTime}")
    }
    // Set goal
    .withGoal(
        "Use this tool to find platform team information including documentation, " +
        "team members, projects, and technical facts. Try both vector and text search " +
        "for comprehensive coverage."
    )

// Expose to LLM
val tools = comprehensiveRag.tools()

Spring AI Integration

Integrate with Spring AI framework for vector store operations.

import com.embabel.agent.rag.service.spring.*
import org.springframework.ai.vectorstore.VectorStore

// Wrap Spring AI VectorStore
val springVectorStore: VectorStore = // Spring AI configuration

val ragSearch = SpringVectorStoreVectorSearch(springVectorStore)

// Use with ToolishRag
val ragTool = ToolishRag(
    name = "spring_search",
    description = "Search using Spring AI vector store",
    searchOperations = ragSearch
)

Filter Conversion

import com.embabel.agent.rag.filter.*
import com.embabel.agent.rag.service.spring.*
import org.springframework.ai.vectorstore.filter.Filter

// Create RAG filter
val ragFilter = PropertyFilter.eq("status", "active")
    .and(PropertyFilter.gte("priority", 5))
    .and(PropertyFilter.`in`("team", "platform", "infrastructure"))

// Convert to Spring AI expression
val springExpression = ragFilter.toSpringAiExpression()

// Use directly with Spring AI VectorStore
val request = SearchRequest.query("search text")
    .withFilterExpression(springExpression)
    .withTopK(10)

val documents = springVectorStore.similaritySearch(request)

Spring Configuration

import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import org.springframework.ai.vectorstore.VectorStore
import com.embabel.agent.rag.service.spring.SpringVectorStoreVectorSearch
import com.embabel.agent.rag.tools.ToolishRag

@Configuration
class RagConfiguration {

    @Bean
    fun ragVectorSearch(springVectorStore: VectorStore): FilteringVectorSearch {
        return SpringVectorStoreVectorSearch(springVectorStore)
    }

    @Bean
    fun documentationSearch(ragVectorSearch: FilteringVectorSearch): ToolishRag {
        return ToolishRag(
            name = "docs_search",
            description = "Search technical documentation",
            searchOperations = ragVectorSearch
        ).withHint(TryHyDE.usingConversationContext())
    }

    @Bean
    fun llmTools(
        documentationSearch: ToolishRag,
        entitySearch: ToolishRag
    ): List<Tool> {
        return documentationSearch.tools() + entitySearch.tools()
    }
}

Multi-Repository Setup

Configure multiple RAG tools for different data sources.

// Documentation repository
val docsRag = ToolishRag(
    name = "documentation",
    description = "Search technical documentation and guides",
    searchOperations = docsRepository
).withMetadataFilter(
    PropertyFilter.eq("type", "documentation")
)

// Code examples repository
val codeRag = ToolishRag(
    name = "code_examples",
    description = "Search code snippets and examples",
    searchOperations = codeRepository
).withMetadataFilter(
    PropertyFilter.eq("type", "code")
).withHint(
    TryHyDE.withContext("Code examples and implementation patterns")
)

// Entity repository
val entityRag = ToolishRag(
    name = "entities",
    description = "Search people, projects, and teams",
    searchOperations = entityRepository
).withSearchFor(
    vectorSearchFor = listOf(NamedEntity::class.java),
    textSearchFor = listOf(NamedEntity::class.java)
).withEntityFilter(
    EntityFilter.hasAnyLabel("Person", "Project", "Team")
)

// Combine all tools
val allTools = docsRag.tools() + codeRag.tools() + entityRag.tools()
println("Total tools available to LLM: ${allTools.size}")

Advanced Patterns

Contextual Tool Selection

// Create tools with different configurations for different contexts
fun createContextualRag(context: String): ToolishRag {
    return when (context) {
        "security" -> ToolishRag(
            name = "security_search",
            description = "Search security documentation and guidelines",
            searchOperations = searchOperations
        ).withMetadataFilter(
            PropertyFilter.eq("category", "security")
        )

        "api" -> ToolishRag(
            name = "api_search",
            description = "Search API documentation and examples",
            searchOperations = searchOperations
        ).withMetadataFilter(
            PropertyFilter.eq("category", "api")
        ).withGoal("Find API endpoints, parameters, and response formats")

        else -> ToolishRag(
            name = "general_search",
            description = "Search all documentation",
            searchOperations = searchOperations
        )
    }
}

Dynamic Filtering

// Create RAG tool with dynamic filter updates
class DynamicRagTool(
    private val baseRag: ToolishRag
) {
    private var currentFilter: PropertyFilter? = null

    fun updateFilter(filter: PropertyFilter?) {
        currentFilter = filter
    }

    fun getTools(): List<Tool> {
        return if (currentFilter != null) {
            baseRag.withMetadataFilter(currentFilter!!).tools()
        } else {
            baseRag.tools()
        }
    }
}

// Use
val dynamicRag = DynamicRagTool(baseRag)

// Update filter based on context
dynamicRag.updateFilter(
    PropertyFilter.eq("department", "engineering")
)

val tools = dynamicRag.getTools()

Result Caching

import java.time.Duration
import java.time.Instant

class CachedRagTool(
    private val baseRag: ToolishRag,
    private val cacheDuration: Duration = Duration.ofMinutes(5)
) {
    private data class CacheEntry(
        val results: String,
        val timestamp: Instant
    )

    private val cache = mutableMapOf<String, CacheEntry>()

    fun getToolsWithCache(): List<Tool> {
        // Wrap the original tools with caching logic
        return baseRag.tools()
    }

    fun clearCache() {
        cache.clear()
    }

    fun cleanExpired() {
        val now = Instant.now()
        cache.entries.removeIf { (_, entry) ->
            Duration.between(entry.timestamp, now) > cacheDuration
        }
    }
}

Edge Cases

Empty Search Results

// Add listener to handle empty results
val gracefulRag = ToolishRag(
    name = "graceful_search",
    description = "Search with fallback handling",
    searchOperations = searchOperations
).withListener { event ->
    if (event.results.isEmpty()) {
        println("Warning: No results for query '${event.query}'")
        println("Consider using text search or broadening the query")
    }
}

Large Result Sets

// Limit result formatting for large sets
val efficientFormatter = RetrievableResultsFormatter { results ->
    val maxResults = 10
    val displayResults = results.results.take(maxResults)

    buildString {
        appendLine("Showing ${displayResults.size} of ${results.results.size} results:")
        appendLine()

        displayResults.forEachIndexed { index, result ->
            appendLine("${index + 1}. ${result.content.infoString(verbose = false)}")
        }

        if (results.results.size > maxResults) {
            appendLine()
            appendLine("... and ${results.results.size - maxResults} more results")
        }
    }
}

Filter Conflicts

// Validate filters before creating tool
fun createValidatedRag(
    name: String,
    description: String,
    searchOperations: SearchOperations,
    metadataFilter: PropertyFilter?
): ToolishRag? {

    // Validate filter logic
    if (metadataFilter != null) {
        try {
            // Test filter (implementation-specific)
            println("Validating filter: $metadataFilter")
        } catch (e: Exception) {
            println("Invalid filter: ${e.message}")
            return null
        }
    }

    return ToolishRag(
        name = name,
        description = description,
        searchOperations = searchOperations
    ).apply {
        if (metadataFilter != null) {
            withMetadataFilter(metadataFilter)
        }
    }
}

Next Steps

Basic RAG Pipeline - Complete end-to-end setup
Vector Search - Advanced search techniques
Entity Management - Work with structured entities

tessl i tessl/maven-com-embabel-agent--embabel-agent-rag-core@0.3.1

docs

advanced

api-reference

quickstart

basic-rag-pipeline.md

utilities

tessl/maven-com-embabel-agent--embabel-agent-rag-core

llm-integration.md.css-3qkkll{font-size:var(--chakra-font-sizes-sm);font-weight:var(--chakra-font-weights-normal);color:var(--chakra-colors-gray-300);}docs/quickstart/

Quickstart: LLM Integration

Overview

Basic ToolishRag Setup

Understanding Generated Tools

Configure Search Types

Type-Specific Tools

Apply Filters

Entity Filters

Combined Filters

Add Search Hints

Custom Context HyDE

Multiple Hints

Custom Result Formatting

Compact Formatter

Monitor Search Events

Analytics Listener

Customize Tool Behavior

Custom Goal

Complete Configuration

Spring AI Integration

Filter Conversion

Spring Configuration

Multi-Repository Setup

Advanced Patterns

Contextual Tool Selection

Dynamic Filtering

Result Caching

Edge Cases

Empty Search Results

Large Result Sets

Filter Conflicts

Next Steps

llm-integration.mddocs/quickstart/