CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

tessl/maven-com-embabel-agent--embabel-agent-common

Common AI framework utilities for the Embabel Agent system including LLM configuration, output converters, prompt contributors, and embedding service abstractions.

Overview
Eval results
Files

model-configuration.mddocs/core/

Model Configuration

Complete LLM configuration system for model selection, hyperparameters, thinking configuration, and timeouts.

Core Concepts

LlmOptions: Main configuration class combining model selection with hyperparameters ModelSelectionCriteria: Type-safe strategy for selecting models Thinking: Configuration for LLM reasoning/thinking capabilities

LLM Options

Complete LLM configuration with fluent builder API.

data class LlmOptions @JvmOverloads constructor(
    var modelSelectionCriteria: ModelSelectionCriteria? = null,
    var model: String? = null,
    var role: String? = null,
    override var temperature: Double? = null,
    override var frequencyPenalty: Double? = null,
    override var maxTokens: Int? = null,
    override var presencePenalty: Double? = null,
    override var topK: Int? = null,
    override var topP: Double? = null,
    var thinking: Thinking? = null,
    var timeout: Duration? = null
) : LlmHyperparameters, HasInfoString {
    val criteria: ModelSelectionCriteria
    fun withTemperature(temperature: Double?): LlmOptions
    fun withMaxTokens(maxTokens: Int): LlmOptions
    fun withTopK(topK: Int): LlmOptions
    fun withTopP(topP: Double): LlmOptions
    fun withFrequencyPenalty(frequencyPenalty: Double): LlmOptions
    fun withPresencePenalty(presencePenalty: Double): LlmOptions
    fun withThinking(thinking: Thinking): LlmOptions
    fun withoutThinking(): LlmOptions
    fun withTimeout(timeout: Duration): LlmOptions

    companion object {
        @JvmStatic operator fun invoke(): LlmOptions
        @JvmStatic operator fun invoke(model: String): LlmOptions
        @JvmStatic operator fun invoke(criteria: ModelSelectionCriteria): LlmOptions
        @JvmStatic fun withDefaults(): LlmOptions
        @JvmStatic fun withModel(model: String): LlmOptions
        @JvmStatic fun withDefaultLlm(): LlmOptions
        @JvmStatic fun withAutoLlm(): LlmOptions
        @JvmStatic fun withLlmForRole(role: String): LlmOptions
        @JvmStatic fun withFirstAvailableLlmOf(vararg names: String): LlmOptions
        @JvmStatic fun fromCriteria(criteria: ModelSelectionCriteria): LlmOptions
    }
}

Factory Methods

Specific model:

val options = LlmOptions.withModel("gpt-4")

Automatic selection:

val options = LlmOptions.withAutoLlm()

Platform default:

val options = LlmOptions.withDefaultLlm()

Role-based selection:

val options = LlmOptions.withLlmForRole("code-generation")

Fallback chain:

val options = LlmOptions.withFirstAvailableLlmOf("gpt-4", "claude-3-opus", "llama-3")

From criteria:

val criteria = ModelSelectionCriteria.byName("gpt-4")
val options = LlmOptions.fromCriteria(criteria)

Hyperparameters

Temperature (0.0 to 1.0+): Controls randomness. Lower = more deterministic, higher = more creative.

options.withTemperature(0.7)

Max tokens: Maximum output length.

options.withMaxTokens(1000)

Top-P (0.0 to 1.0): Nucleus sampling threshold.

options.withTopP(0.9)

Top-K: Consider top K tokens only.

options.withTopK(50)

Frequency penalty (-2.0 to 2.0): Penalizes token frequency.

options.withFrequencyPenalty(0.5)

Presence penalty (-2.0 to 2.0): Penalizes tokens already present.

options.withPresencePenalty(0.3)

Complete Example

val options = LlmOptions.withModel("gpt-4")
    .withTemperature(0.8)
    .withMaxTokens(1500)
    .withTopP(0.9)
    .withTopK(50)
    .withFrequencyPenalty(0.5)
    .withPresencePenalty(0.3)
    .withTimeout(Duration.ofSeconds(30))

Selection Strategies

Type-safe strategies for model selection.

sealed interface ModelSelectionCriteria {
    companion object {
        @JvmStatic fun byRole(role: String): ModelSelectionCriteria
        @JvmStatic fun byName(name: String): ModelSelectionCriteria
        @JvmStatic fun randomOf(vararg names: String): ModelSelectionCriteria
        @JvmStatic fun firstOf(vararg names: String): ModelSelectionCriteria
        @JvmStatic val Auto: ModelSelectionCriteria
        @JvmStatic val PlatformDefault: ModelSelectionCriteria
    }
}

Strategy Types

By exact name:

val criteria = ModelSelectionCriteria.byName("gpt-4")

By role (application-defined roles):

val criteria = ModelSelectionCriteria.byRole("code-reviewer")

Random selection (load balancing):

val criteria = ModelSelectionCriteria.randomOf("gpt-4", "claude-3-opus")

Fallback chain (try in order until one works):

val criteria = ModelSelectionCriteria.firstOf("gpt-4-turbo", "gpt-4", "gpt-3.5-turbo")

Automatic (platform decides):

val criteria = ModelSelectionCriteria.Auto

Platform default:

val criteria = ModelSelectionCriteria.PlatformDefault

Concrete Types

data class ByNameModelSelectionCriteria(val name: String) : ModelSelectionCriteria
data class ByRoleModelSelectionCriteria(val role: String) : ModelSelectionCriteria
data class RandomByNameModelSelectionCriteria(val names: List<String>) : ModelSelectionCriteria
data class FallbackByNameModelSelectionCriteria(val names: List<String>) : ModelSelectionCriteria
object AutoModelSelectionCriteria : ModelSelectionCriteria
object DefaultModelSelectionCriteria : ModelSelectionCriteria

Thinking Configuration

Configuration for LLM reasoning with token budgets and extraction.

class Thinking private constructor(
    val enabled: Boolean = false,
    val tokenBudget: Int? = null,
    val extractThinking: Boolean = false
) {
    fun applyExtraction(): Thinking
    fun applyTokenBudget(tokenBudget: Int): Thinking

    companion object {
        @JvmStatic fun withTokenBudget(tokenBudget: Int): Thinking
        @JvmStatic fun withExtraction(): Thinking
        @JvmStatic val NONE: Thinking
    }
}

Usage

With token budget:

val thinking = Thinking.withTokenBudget(5000)
val options = LlmOptions.withModel("o1-preview").withThinking(thinking)

With extraction:

val thinking = Thinking.withExtraction()
val options = LlmOptions.withModel("o1-preview").withThinking(thinking)

Disabled:

val options = LlmOptions.withModel("gpt-4").withoutThinking()

Combined:

val thinking = Thinking.withTokenBudget(3000).applyExtraction()

Hyperparameters Interface

Common hyperparameters interface implemented by LlmOptions.

interface LlmHyperparameters {
    val temperature: Double?
    val frequencyPenalty: Double?
    val maxTokens: Int?
    val presencePenalty: Double?
    val topK: Int?
    val topP: Double?
}

Common Patterns

Development vs Production

// Development: higher creativity, verbose
val devOptions = LlmOptions.withModel("gpt-4")
    .withTemperature(0.9)
    .withMaxTokens(2000)

// Production: deterministic, concise
val prodOptions = LlmOptions.withModel("gpt-4")
    .withTemperature(0.2)
    .withMaxTokens(500)

Multi-Model Strategy

class ModelStrategy {
    fun forTask(task: String): LlmOptions = when (task) {
        "code-generation" -> LlmOptions.withModel("gpt-4")
            .withTemperature(0.7)
        "summarization" -> LlmOptions.withModel("gpt-3.5-turbo")
            .withTemperature(0.3)
        "creative-writing" -> LlmOptions.withModel("claude-3-opus")
            .withTemperature(0.9)
        else -> LlmOptions.withDefaultLlm()
    }
}

Timeout Configuration

// Quick responses
val quickOptions = LlmOptions.withModel("gpt-3.5-turbo")
    .withTimeout(Duration.ofSeconds(5))

// Complex tasks
val complexOptions = LlmOptions.withModel("gpt-4")
    .withTimeout(Duration.ofSeconds(60))

When to Use What

TaskModel SelectionTemperatureMax Tokens
Code generationSpecific model0.71000-2000
Data extractionSpecific model0.2500
Creative writingRandom/fallback0.92000+
SummarizationBy role0.3500
ClassificationSpecific model0.050
Chat/conversationAuto/default0.81000

→ See Performance Guide for optimization tips

tessl i tessl/maven-com-embabel-agent--embabel-agent-common@0.3.1

docs

core

model-configuration.md

output-converters.md

streaming.md

index.md

quick-reference.md

README.md

tile.json