CtrlK

Community Documentation Log in Get started

tessl/maven-com-embabel-agent--embabel-agent-rag-core

RAG (Retrieval-Augmented Generation) framework for the Embabel Agent platform providing content ingestion, chunking, hierarchical navigation, and semantic search capabilities

Overview

Eval results

Files

Quickstart: Entity Management

Name: tessl/maven-com-embabel-agent--embabel-agent-rag-core
Author: tessl

This guide covers working with named entities including creation, relationships, search, and type conversion.

Overview

Named entities represent structured objects like people, projects, organizations, or any domain-specific concepts with:

Properties - Arbitrary key-value attributes
Relationships - Named connections to other entities
Search - Vector and text search capabilities
Type conversion - Dynamic proxies and typed instances

Basic Entity Operations

Create and Save Entities

import com.embabel.agent.rag.service.NamedEntityDataRepository
import com.embabel.agent.rag.model.*

val entityRepo: NamedEntityDataRepository = // implementation

// Create a simple entity
val person = SimpleNamedEntityData(
    id = "person-123",
    name = "Alice Smith",
    description = "Senior software engineer specializing in distributed systems",
    properties = mapOf(
        "role" to "engineer",
        "team" to "platform",
        "yearsExperience" to 8,
        "skills" to listOf("Kotlin", "Java", "Python", "Go"),
        "email" to "alice@example.com",
        "location" to "San Francisco"
    )
)

// Save entity
val saved = entityRepo.save(person)
println("Saved entity: ${saved.name}")

Find Entities

// Find by ID
val found = entityRepo.findById("person-123")
if (found != null) {
    println("Name: ${found.name}")
    println("Description: ${found.description}")
    println("Properties: ${found.properties}")
}

// Find all entities with a label
val allPeople = entityRepo.findByLabel("Person")
println("Found ${allPeople.size} people")

Update Entities

// Update properties
val updated = saved.copy(
    properties = saved.properties + mapOf(
        "level" to "senior",
        "certifications" to listOf("AWS", "Kubernetes")
    )
)

entityRepo.update(updated)
println("Updated entity with new properties")

Delete Entities

// Delete by ID
val deleted = entityRepo.delete("person-123")
if (deleted) {
    println("Entity deleted successfully")
} else {
    println("Entity not found")
}

Batch Operations

Save multiple entities efficiently.

// Create multiple entities
val entities = listOf(
    SimpleNamedEntityData(
        id = "person-1",
        name = "Alice",
        description = "Software Engineer",
        properties = mapOf(
            "team" to "platform",
            "role" to "engineer"
        )
    ),
    SimpleNamedEntityData(
        id = "person-2",
        name = "Bob",
        description = "Product Designer",
        properties = mapOf(
            "team" to "product",
            "role" to "designer"
        )
    ),
    SimpleNamedEntityData(
        id = "person-3",
        name = "Carol",
        description = "Engineering Manager",
        properties = mapOf(
            "team" to "platform",
            "role" to "manager"
        )
    )
)

// Batch save
val savedEntities = entityRepo.saveAll(entities)
println("Saved ${savedEntities.size} entities")

Working with Relationships

Create Relationships

import com.embabel.agent.rag.service.*
import com.embabel.agent.rag.model.*

// Create entities
val alice = SimpleNamedEntityData(
    id = "person-1",
    name = "Alice",
    description = "Senior Engineer"
)

val bob = SimpleNamedEntityData(
    id = "person-2",
    name = "Bob",
    description = "Engineer"
)

val project = SimpleNamedEntityData(
    id = "project-1",
    name = "Platform Rewrite",
    description = "Major platform refactoring initiative"
)

// Save entities
entityRepo.saveAll(listOf(alice, bob, project))

// Create relationships
entityRepo.createRelationship(
    a = RetrievableIdentifier("person-1", "Person"),
    b = RetrievableIdentifier("person-2", "Person"),
    relationship = RelationshipData(
        name = "WORKS_WITH",
        properties = mapOf(
            "since" to "2024-01-01",
            "collaboration" to "high"
        )
    )
)

entityRepo.createRelationship(
    a = RetrievableIdentifier("person-1", "Person"),
    b = RetrievableIdentifier("project-1", "Project"),
    relationship = RelationshipData(
        name = "CONTRIBUTES_TO",
        properties = mapOf(
            "role" to "lead",
            "startDate" to "2024-01-15"
        )
    )
)

entityRepo.createRelationship(
    a = RetrievableIdentifier("person-2", "Person"),
    b = RetrievableIdentifier("project-1", "Project"),
    relationship = RelationshipData(
        name = "CONTRIBUTES_TO",
        properties = mapOf(
            "role" to "contributor",
            "startDate" to "2024-02-01"
        )
    )
)

Navigate Relationships

// Find outgoing relationships
val colleagues = entityRepo.findRelated(
    source = RetrievableIdentifier("person-1", "Person"),
    relationshipName = "WORKS_WITH",
    direction = RelationshipDirection.OUTGOING
)

println("Alice works with:")
colleagues.forEach { colleague ->
    println("  - ${colleague.name}")
}

// Find projects
val projects = entityRepo.findRelated(
    source = RetrievableIdentifier("person-1", "Person"),
    relationshipName = "CONTRIBUTES_TO",
    direction = RelationshipDirection.OUTGOING
)

println("Alice contributes to:")
projects.forEach { project ->
    println("  - ${project.name}")
}

// Find incoming relationships (who contributes to a project)
val contributors = entityRepo.findRelated(
    source = RetrievableIdentifier("project-1", "Project"),
    relationshipName = "CONTRIBUTES_TO",
    direction = RelationshipDirection.INCOMING
)

println("Project contributors:")
contributors.forEach { contributor ->
    println("  - ${contributor.name}")
}

Find Single Related Entity

// Find manager (expecting single result)
val manager = entityRepo.findRelatedSingle(
    source = RetrievableIdentifier("person-1", "Person"),
    relationshipName = "REPORTS_TO",
    direction = RelationshipDirection.OUTGOING
)

if (manager != null) {
    println("Reports to: ${manager.name}")
} else {
    println("No manager found")
}

Typed Entity Interfaces

Define strongly-typed interfaces for entities.

// Define entity interface
interface Employee : NamedEntity {
    val role: String
    val team: String
    val yearsExperience: Int
    val skills: List<String>
}

// Convert to typed instance
val entityData = entityRepo.findById("person-123")!!
val employee = entityData.toTypedInstance<Employee>(
    objectMapper = entityRepo.objectMapper,
    type = Employee::class.java
)

if (employee != null) {
    println("Employee: ${employee.name}")
    println("Role: ${employee.role}")
    println("Team: ${employee.team}")
    println("Experience: ${employee.yearsExperience} years")
    println("Skills: ${employee.skills.joinToString()}")
}

// Find all employees
val allEmployees = entityRepo.findAll(Employee::class.java)
allEmployees.forEach { emp ->
    println("${emp.name} - ${emp.role} (${emp.team})")
}

Relationship Annotations

Use annotations to define relationship navigation methods.

import com.embabel.agent.rag.model.*

// Define entity with relationship methods
interface Person : NamedEntity {
    @Relationship(name = "WORKS_WITH", direction = RelationshipDirection.OUTGOING)
    fun getColleagues(): List<NamedEntity>

    @Relationship(name = "CONTRIBUTES_TO", direction = RelationshipDirection.OUTGOING)
    fun getProjects(): List<NamedEntity>

    @Relationship(name = "REPORTS_TO", direction = RelationshipDirection.OUTGOING)
    fun getManager(): NamedEntity?

    @Relationship(name = "MANAGES", direction = RelationshipDirection.OUTGOING)
    fun getDirectReports(): List<NamedEntity>
}

// Load entity as dynamic proxy
val entityData = entityRepo.findById("person-1")!!
val person = entityData.toInstance<Person>(
    navigator = entityRepo,
    Person::class.java
)

// Navigate relationships via methods
val colleagues = person.getColleagues()
println("Colleagues: ${colleagues.map { it.name }}")

val projects = person.getProjects()
println("Projects: ${projects.map { it.name }}")

val manager = person.getManager()
println("Manager: ${manager?.name ?: "None"}")

val reports = person.getDirectReports()
println("Direct reports: ${reports.map { it.name }}")

Custom Relationship Names

interface Project : NamedEntity {
    // Use custom relationship name
    @Relationship(name = "HAS_CONTRIBUTOR", direction = RelationshipDirection.INCOMING)
    fun getTeamMembers(): List<NamedEntity>

    @Relationship(name = "DEPENDS_ON", direction = RelationshipDirection.OUTGOING)
    fun getDependencies(): List<NamedEntity>

    @Relationship(name = "BLOCKS", direction = RelationshipDirection.OUTGOING)
    fun getBlockedProjects(): List<NamedEntity>
}

Searching Entities

Vector Search

import com.embabel.agent.rag.filter.*

// Basic vector search
val results = entityRepo.vectorSearch(
    request = TextSimilaritySearchRequest(
        query = "experienced platform engineer",
        topK = 10,
        similarityThreshold = 0.7
    )
)

results.forEach { result ->
    println("Score: ${"%.3f".format(result.score)}")
    println("Name: ${result.content.name}")
    println("Description: ${result.content.description}")
    println()
}

// Vector search with filters
val filteredResults = entityRepo.vectorSearch(
    request = TextSimilaritySearchRequest(
        query = "software engineer",
        topK = 10
    ),
    metadataFilter = PropertyFilter.eq("team", "platform")
        .and(PropertyFilter.gte("yearsExperience", 5)),
    entityFilter = EntityFilter.hasAnyLabel("Person", "Employee")
)

Text Search

// Check Lucene syntax support
println("Syntax notes: ${entityRepo.luceneSyntaxNotes}")

// Text search with Lucene query
val textResults = entityRepo.textSearch(
    request = TextSimilaritySearchRequest(
        query = "engineer AND (platform OR infrastructure)",
        topK = 20
    )
)

// Text search with filters
val filteredTextResults = entityRepo.textSearch(
    request = TextSimilaritySearchRequest(
        query = "manager",
        topK = 10
    ),
    metadataFilter = PropertyFilter.contains("team", "platform")
)

Filtered Queries

// Find entities by label
val allPeople = entityRepo.findByLabel("Person")

// Find with property filter
val platformEngineers = entityRepo.find(
    label = "Person",
    filter = PropertyFilter.eq("team", "platform")
        .and(PropertyFilter.eq("role", "engineer"))
)

// Find with multiple labels
val teamMembers = entityRepo.find(
    labels = EntityFilter.hasAnyLabel("Person", "Contractor"),
    filter = PropertyFilter.eq("team", "platform")
)

Domain Types

Link entities to domain type definitions.

// Assuming you have a DomainType defined
val personType: DomainType = // from data dictionary

// Find entities by domain type
val people = entityRepo.findByDomainType<NamedEntity>(personType)
println("Found ${people.size} entities of type ${personType.name}")

// Get entity data
val entityData = entityRepo.findEntityDataByDomainType(personType)
entityData.forEach { data ->
    println("${data.name}: ${data.properties}")
}

// Create entity with linked domain type
val linkedEntity = SimpleNamedEntityData(
    id = "person-456",
    name = "David",
    description = "Data Scientist",
    properties = mapOf("specialty" to "machine learning"),
    linkedDomainType = personType
)

entityRepo.save(linkedEntity)

Type Checking

Check support for different entity types.

// Check if type is supported
if (entityRepo.supportsType("Person")) {
    val person = entityRepo.findById<NamedEntity>("person-123", "Person")
    println("Found person: ${person?.name}")
}

// Check native type support
if (entityRepo.isNativeType(CustomEntity::class.java)) {
    val entity = entityRepo.findNativeById("id-123", CustomEntity::class.java)
    println("Found native entity: ${entity?.name}")
}

// Find all native entities of a type
val nativeEntities = entityRepo.findNativeAll(CustomEntity::class.java)
nativeEntities?.forEach { entity ->
    println("Native entity: ${entity.name}")
}

Common Patterns

Organization Hierarchy

// Create org structure
val ceo = SimpleNamedEntityData(
    id = "person-ceo",
    name = "CEO",
    description = "Chief Executive Officer",
    properties = mapOf("level" to 1)
)

val vp = SimpleNamedEntityData(
    id = "person-vp",
    name = "VP Engineering",
    description = "Vice President of Engineering",
    properties = mapOf("level" to 2)
)

val manager = SimpleNamedEntityData(
    id = "person-manager",
    name = "Engineering Manager",
    description = "Platform Team Manager",
    properties = mapOf("level" to 3)
)

val engineer = SimpleNamedEntityData(
    id = "person-engineer",
    name = "Senior Engineer",
    description = "Platform Engineer",
    properties = mapOf("level" to 4)
)

entityRepo.saveAll(listOf(ceo, vp, manager, engineer))

// Create hierarchy
entityRepo.createRelationship(
    RetrievableIdentifier("person-vp", "Person"),
    RetrievableIdentifier("person-ceo", "Person"),
    RelationshipData("REPORTS_TO")
)

entityRepo.createRelationship(
    RetrievableIdentifier("person-manager", "Person"),
    RetrievableIdentifier("person-vp", "Person"),
    RelationshipData("REPORTS_TO")
)

entityRepo.createRelationship(
    RetrievableIdentifier("person-engineer", "Person"),
    RetrievableIdentifier("person-manager", "Person"),
    RelationshipData("REPORTS_TO")
)

// Navigate upward
fun findManagerChain(entityRepo: NamedEntityDataRepository, personId: String): List<String> {
    val chain = mutableListOf<String>()
    var current: NamedEntityData? = entityRepo.findById(personId)

    while (current != null) {
        chain.add(current.name)
        current = entityRepo.findRelatedSingle(
            RetrievableIdentifier(current.id, "Person"),
            "REPORTS_TO",
            RelationshipDirection.OUTGOING
        )
    }

    return chain
}

val chain = findManagerChain(entityRepo, "person-engineer")
println("Management chain: ${chain.joinToString(" → ")}")

Project Dependencies

// Create projects with dependencies
val projectA = SimpleNamedEntityData(
    id = "project-a",
    name = "Project A",
    description = "Core platform"
)

val projectB = SimpleNamedEntityData(
    id = "project-b",
    name = "Project B",
    description = "API service",
    properties = mapOf("dependsOn" to listOf("project-a"))
)

val projectC = SimpleNamedEntityData(
    id = "project-c",
    name = "Project C",
    description = "UI application",
    properties = mapOf("dependsOn" to listOf("project-b"))
)

entityRepo.saveAll(listOf(projectA, projectB, projectC))

// Create dependency relationships
entityRepo.createRelationship(
    RetrievableIdentifier("project-b", "Project"),
    RetrievableIdentifier("project-a", "Project"),
    RelationshipData("DEPENDS_ON")
)

entityRepo.createRelationship(
    RetrievableIdentifier("project-c", "Project"),
    RetrievableIdentifier("project-b", "Project"),
    RelationshipData("DEPENDS_ON")
)

// Find all dependencies recursively
fun findAllDependencies(
    entityRepo: NamedEntityDataRepository,
    projectId: String,
    visited: MutableSet<String> = mutableSetOf()
): Set<String> {
    if (projectId in visited) return emptySet()
    visited.add(projectId)

    val dependencies = entityRepo.findRelated(
        RetrievableIdentifier(projectId, "Project"),
        "DEPENDS_ON",
        RelationshipDirection.OUTGOING
    )

    val allDeps = mutableSetOf<String>()
    dependencies.forEach { dep ->
        allDeps.add(dep.id)
        allDeps.addAll(findAllDependencies(entityRepo, dep.id, visited))
    }

    return allDeps
}

val allDeps = findAllDependencies(entityRepo, "project-c")
println("All dependencies for Project C: $allDeps")

Skills and Expertise Graph

// Create skill entities
val kotlin = SimpleNamedEntityData(
    id = "skill-kotlin",
    name = "Kotlin",
    description = "Modern JVM language",
    properties = mapOf("category" to "programming")
)

val springBoot = SimpleNamedEntityData(
    id = "skill-spring",
    name = "Spring Boot",
    description = "Java application framework",
    properties = mapOf("category" to "framework")
)

entityRepo.saveAll(listOf(kotlin, springBoot))

// Link people to skills
entityRepo.createRelationship(
    RetrievableIdentifier("person-1", "Person"),
    RetrievableIdentifier("skill-kotlin", "Skill"),
    RelationshipData(
        name = "HAS_SKILL",
        properties = mapOf("level" to "expert", "years" to 5)
    )
)

// Find people by skill
val kotlinExperts = entityRepo.findRelated(
    RetrievableIdentifier("skill-kotlin", "Skill"),
    "HAS_SKILL",
    RelationshipDirection.INCOMING
)

println("Kotlin experts: ${kotlinExperts.map { it.name }}")

Edge Cases

Circular Relationships

// Handle circular dependencies
fun findRelatedSafe(
    entityRepo: NamedEntityDataRepository,
    entityId: String,
    relationshipName: String,
    visited: MutableSet<String> = mutableSetOf()
): List<NamedEntityData> {
    if (entityId in visited) {
        return emptyList()
    }
    visited.add(entityId)

    return entityRepo.findRelated(
        RetrievableIdentifier(entityId, "Entity"),
        relationshipName,
        RelationshipDirection.BOTH
    )
}

Missing Entities

// Safely handle missing entities
val entity = entityRepo.findById("unknown-id")
if (entity == null) {
    println("Entity not found")
    // Handle appropriately
} else {
    println("Found: ${entity.name}")
}

Invalid Property Types

// Validate properties before saving
fun validateAndSave(
    entityRepo: NamedEntityDataRepository,
    entity: SimpleNamedEntityData
): SimpleNamedEntityData? {

    // Check required properties
    if (!entity.properties.containsKey("team")) {
        println("Error: 'team' property is required")
        return null
    }

    // Validate property types
    val yearsExp = entity.properties["yearsExperience"]
    if (yearsExp != null && yearsExp !is Number) {
        println("Error: 'yearsExperience' must be a number")
        return null
    }

    return entityRepo.save(entity)
}

Next Steps

LLM Integration - Expose entities to language models
Vector Search - Advanced entity search
Basic RAG Pipeline - Complete system integration

tessl i tessl/maven-com-embabel-agent--embabel-agent-rag-core@0.3.1

docs

advanced

api-reference

quickstart

basic-rag-pipeline.md

utilities

tessl/maven-com-embabel-agent--embabel-agent-rag-core

entity-management.md.css-3qkkll{font-size:var(--chakra-font-sizes-sm);font-weight:var(--chakra-font-weights-normal);color:var(--chakra-colors-gray-300);}docs/quickstart/

Quickstart: Entity Management

Overview

Basic Entity Operations

Create and Save Entities

Find Entities

Update Entities

Delete Entities

Batch Operations

Working with Relationships

Create Relationships

Navigate Relationships

Find Single Related Entity

Typed Entity Interfaces

Relationship Annotations

Custom Relationship Names

Searching Entities

Vector Search

Text Search

Filtered Queries

Domain Types

Type Checking

Common Patterns

Organization Hierarchy

Project Dependencies

Skills and Expertise Graph

Edge Cases

Circular Relationships

Missing Entities

Invalid Property Types

Next Steps

entity-management.mddocs/quickstart/