CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

tessl/maven-com-embabel-agent--embabel-agent-test-support

Multi-module test support framework for Embabel Agent applications providing integration testing, mock AI services, and test configuration utilities

Overview
Eval results
Files

testing-options-converters.mddocs/guides/

Testing Options Converters Guide

Step-by-step guide for testing custom options converters that preserve core LLM values.

What is an Options Converter?

An OptionsConverter transforms framework-agnostic LlmOptions into provider-specific ChatOptions (e.g., OpenAI, Anthropic). Testing ensures your converter preserves critical values like temperature, topP, and frequencyPenalty.

Prerequisites

  • embabel-agent-test-internal dependency installed
  • Custom options converter implementation
  • Understanding of LLM parameters

Why Test Options Converters?

Critical LLM parameters must be preserved during conversion:

ParameterPurposeMust Preserve
temperatureControls randomness✓ Yes
topPNucleus sampling✓ Yes
frequencyPenaltyToken repetition penalty✓ Yes
maxTokensOutput limitRecommended

Basic Testing Pattern

Method 1: Using OptionsConverterTestSupport

Extend the base test class for automatic validation.

import com.embabel.agent.test.models.OptionsConverterTestSupport
import org.junit.jupiter.api.Test

class MyOptionsConverterTest : OptionsConverterTestSupport<MyCustomOptions>(
    MyOptionsConverter()
) {
    // Inherits 'should preserve core values' test
    // Automatically tests temperature, topP, frequencyPenalty

    @Test
    fun `should handle custom parameters`() {
        // Add your custom parameter tests
    }
}

Method 2: Using Utility Function

Use the standalone function for more flexible testing.

import com.embabel.agent.test.models.checkOptionsConverterPreservesCoreValues
import org.junit.jupiter.api.Test

class MyOptionsConverterTest {

    @Test
    fun `should preserve core values`() {
        val converter = MyOptionsConverter()

        // This throws AssertionError if preservation fails
        checkOptionsConverterPreservesCoreValues(converter)
    }
}

Complete Examples

Example 1: Basic Converter Test (Kotlin)

import com.embabel.agent.test.models.OptionsConverterTestSupport
import com.embabel.common.ai.model.OptionsConverter
import org.springframework.ai.openai.OpenAiChatOptions
import org.junit.jupiter.api.Test
import kotlin.test.assertEquals

class OpenAiOptionsConverterTest : OptionsConverterTestSupport<OpenAiChatOptions>(
    OpenAiOptionsConverter()
) {
    // Core preservation test is inherited

    @Test
    fun `should convert max tokens`() {
        val llmOptions = LlmOptions(
            maxTokens = 1000
        )

        val result = optionsConverter.convert(llmOptions)

        assertEquals(1000, result.maxTokens)
    }

    @Test
    fun `should convert all parameters`() {
        val llmOptions = LlmOptions(
            temperature = 0.7,
            topP = 0.9,
            frequencyPenalty = 0.5,
            maxTokens = 2000
        )

        val result = optionsConverter.convert(llmOptions)

        assertEquals(0.7, result.temperature)
        assertEquals(0.9, result.topP)
        assertEquals(0.5, result.frequencyPenalty)
        assertEquals(2000, result.maxTokens)
    }
}

Example 2: Using Utility Function (Kotlin)

import com.embabel.agent.test.models.checkOptionsConverterPreservesCoreValues
import org.junit.jupiter.api.Test
import kotlin.test.assertEquals

class AnthropicOptionsConverterTest {

    private val converter = AnthropicOptionsConverter()

    @Test
    fun `should preserve core values`() {
        checkOptionsConverterPreservesCoreValues(converter)
    }

    @Test
    fun `should handle Anthropic-specific parameters`() {
        val llmOptions = LlmOptions(
            temperature = 0.5,
            maxTokens = 1500
        )

        val result = converter.convert(llmOptions)

        assertEquals(0.5, result.temperature)
        assertEquals(1500, result.maxTokensToSample)
    }
}

Example 3: Java Version

import com.embabel.agent.test.models.OptionsConverterTestSupport;
import static com.embabel.agent.test.models.OptionsConverterTestUtilsKt.checkOptionsConverterPreservesCoreValues;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;

public class OpenAiOptionsConverterTest {

    private final OpenAiOptionsConverter converter = new OpenAiOptionsConverter();

    @Test
    public void shouldPreserveCoreValues() {
        checkOptionsConverterPreservesCoreValues(converter);
    }

    @Test
    public void shouldConvertAllParameters() {
        LlmOptions llmOptions = new LlmOptions();
        llmOptions.setTemperature(0.7);
        llmOptions.setTopP(0.9);
        llmOptions.setFrequencyPenalty(0.5);
        llmOptions.setMaxTokens(2000);

        OpenAiChatOptions result = converter.convert(llmOptions);

        assertEquals(0.7, result.getTemperature(), 0.001);
        assertEquals(0.9, result.getTopP(), 0.001);
        assertEquals(0.5, result.getFrequencyPenalty(), 0.001);
        assertEquals(2000, result.getMaxTokens());
    }
}

Testing Custom Parameters

Provider-Specific Parameters

Test parameters unique to your provider:

@Test
fun `should handle OpenAI presence penalty`() {
    val llmOptions = LlmOptions(
        temperature = 0.7
    )
    // Add custom parameter
    llmOptions.setCustomParameter("presencePenalty", 0.6)

    val result = optionsConverter.convert(llmOptions)

    assertEquals(0.6, result.presencePenalty)
}

Default Values

Test that defaults are applied correctly:

@Test
fun `should apply default values`() {
    val llmOptions = LlmOptions()  // Empty options

    val result = optionsConverter.convert(llmOptions)

    // Verify defaults
    assertEquals(0.7, result.temperature)  // Default temperature
    assertNull(result.maxTokens)  // No default for tokens
}

Null Handling

Test behavior with null values:

@Test
fun `should handle null values`() {
    val llmOptions = LlmOptions(
        temperature = null,
        topP = 0.9,
        frequencyPenalty = null
    )

    val result = optionsConverter.convert(llmOptions)

    assertNull(result.temperature)
    assertEquals(0.9, result.topP)
    assertNull(result.frequencyPenalty)
}

Advanced Testing Patterns

Pattern 1: Range Validation

Test that values are within valid ranges:

@Test
fun `should enforce valid temperature range`() {
    // Test minimum
    var llmOptions = LlmOptions(temperature = 0.0)
    var result = optionsConverter.convert(llmOptions)
    assertEquals(0.0, result.temperature)

    // Test maximum
    llmOptions = LlmOptions(temperature = 2.0)
    result = optionsConverter.convert(llmOptions)
    assertEquals(2.0, result.temperature)
}

Pattern 2: Edge Cases

Test boundary conditions:

@Test
fun `should handle edge cases`() {
    // Very small temperature
    var options = LlmOptions(temperature = 0.0001)
    var result = optionsConverter.convert(options)
    assertEquals(0.0001, result.temperature, 0.00001)

    // Very large max tokens
    options = LlmOptions(maxTokens = 100000)
    result = optionsConverter.convert(options)
    assertEquals(100000, result.maxTokens)
}

Pattern 3: Immutability

Test that conversion doesn't modify original:

@Test
fun `should not modify original options`() {
    val original = LlmOptions(
        temperature = 0.7,
        topP = 0.9
    )

    val converted = optionsConverter.convert(original)

    // Original unchanged
    assertEquals(0.7, original.temperature)
    assertEquals(0.9, original.topP)

    // Converted has same values
    assertEquals(0.7, converted.temperature)
    assertEquals(0.9, converted.topP)
}

Pattern 4: Multiple Conversions

Test converting same options multiple times:

@Test
fun `should be idempotent`() {
    val llmOptions = LlmOptions(temperature = 0.7)

    val result1 = optionsConverter.convert(llmOptions)
    val result2 = optionsConverter.convert(llmOptions)

    assertEquals(result1.temperature, result2.temperature)
    assertEquals(result1.topP, result2.topP)
}

Testing Different Providers

OpenAI Converter

class OpenAiOptionsConverterTest : OptionsConverterTestSupport<OpenAiChatOptions>(
    OpenAiOptionsConverter()
) {
    @Test
    fun `should handle OpenAI specific options`() {
        val options = LlmOptions(
            temperature = 0.8,
            maxTokens = 1000
        )

        val result = optionsConverter.convert(options)

        assertEquals(0.8, result.temperature)
        assertEquals(1000, result.maxTokens)
        assertNotNull(result.model)
    }
}

Anthropic Converter

class AnthropicOptionsConverterTest : OptionsConverterTestSupport<AnthropicChatOptions>(
    AnthropicOptionsConverter()
) {
    @Test
    fun `should handle Anthropic specific options`() {
        val options = LlmOptions(
            temperature = 0.6,
            maxTokens = 2000
        )

        val result = optionsConverter.convert(options)

        assertEquals(0.6, result.temperature)
        assertEquals(2000, result.maxTokensToSample)
    }
}

Custom Provider Converter

class CustomProviderConverterTest : OptionsConverterTestSupport<CustomChatOptions>(
    CustomProviderConverter()
) {
    @Test
    fun `should handle custom provider options`() {
        val options = LlmOptions(
            temperature = 0.5,
            customParam = "value"
        )

        val result = optionsConverter.convert(options)

        assertEquals(0.5, result.temperature)
        assertEquals("value", result.customParameter)
    }
}

What Gets Tested Automatically

When using OptionsConverterTestSupport or checkOptionsConverterPreservesCoreValues, these values are automatically tested:

Temperature

// Automatically tests:
val options = LlmOptions(temperature = 0.7)
val result = converter.convert(options)
assertEquals(0.7, result.temperature)

TopP

// Automatically tests:
val options = LlmOptions(topP = 0.9)
val result = converter.convert(options)
assertEquals(0.9, result.topP)

Frequency Penalty

// Automatically tests:
val options = LlmOptions(frequencyPenalty = 0.5)
val result = converter.convert(options)
assertEquals(0.5, result.frequencyPenalty)

Troubleshooting

Test Fails with AssertionError

Problem: Core preservation test fails.

Solution: Check your converter preserves the values:

override fun convert(options: LlmOptions): MyCustomOptions {
    return MyCustomOptions.builder()
        .withTemperature(options.temperature)  // Must preserve
        .withTopP(options.topP)  // Must preserve
        .withFrequencyPenalty(options.frequencyPenalty)  // Must preserve
        .build()
}

Wrong Generic Type

Problem: Type parameter doesn't match.

Solution: Ensure type parameter matches converter output:

// Correct: matches converter output type
class MyTest : OptionsConverterTestSupport<OpenAiChatOptions>(
    MyConverter()  // Produces OpenAiChatOptions
)

// Wrong: type mismatch
class MyTest : OptionsConverterTestSupport<AnthropicChatOptions>(
    MyConverter()  // Produces OpenAiChatOptions - WRONG!
)

Floating Point Precision

Problem: Floating point comparison fails.

Solution: Use delta for comparison:

assertEquals(0.7, result.temperature, 0.001)  // Allow small difference

Best Practices

  1. Always Test Core Values: Use the provided utilities for temperature, topP, frequencyPenalty

  2. Test Provider-Specific: Add tests for your provider's unique parameters

  3. Test Edge Cases: Include boundary values and null handling

  4. Test Defaults: Verify default values are applied correctly

  5. Keep Tests Simple: One assertion per test when possible

  6. Use Descriptive Names: Test names should describe what's being verified

// Good names
@Test
fun `should preserve temperature when converting to OpenAI options`() { }

@Test
fun `should apply default max tokens of 1000`() { }

@Test
fun `should handle null frequency penalty`() { }

Next Steps

  • Test Configuration API - Complete API reference
  • Test Configuration Module - Module details
  • Spring Test Setup Guide - Configure Spring tests
  • Testing Patterns - More patterns

Related Topics

tessl i tessl/maven-com-embabel-agent--embabel-agent-test-support@0.3.0

docs

guides

spring-test-setup.md

stubbing-llm-calls.md

testing-embeddings.md

testing-options-converters.md

verifying-llm-usage.md

index.md

tile.json