CtrlK

Community Documentation Log in Get started

tessl/maven-com-embabel-agent--embabel-agent-test-support

Multi-module test support framework for Embabel Agent applications providing integration testing, mock AI services, and test configuration utilities

Overview

Eval results

Files

Spring Test Setup Guide

Name: tessl/maven-com-embabel-agent--embabel-agent-test-support
Author: tessl

Step-by-step guide for configuring Spring Boot tests with fake AI services.

What is FakeAiConfiguration?

FakeAiConfiguration is a Spring @TestConfiguration that provides pre-configured fake LLM and embedding service beans. It allows you to test Spring components without requiring API keys or making real API calls.

Prerequisites

embabel-agent-test-internal dependency installed
Spring Boot test framework
Basic understanding of Spring dependency injection

Basic Setup Pattern

Step 1: Add Configuration Import

@SpringBootTest
@Import(FakeAiConfiguration::class)  // Add this
class MyTest {
    // Tests go here
}

Step 2: Inject Beans

@Autowired
private lateinit var cheapest: LlmService<*>

@Autowired
private lateinit var best: LlmService<*>

@Autowired
private lateinit var embeddingService: EmbeddingService

Step 3: Use in Tests

@Test
fun `test with fake services`() {
    val result = myService.process(input, cheapest)
    assertNotNull(result)
}

Available Beans

Cheapest LLM Service

Pre-configured fake LLM service simulating gpt-4o-mini.

@Autowired
@Qualifier("cheapest")
private lateinit var cheapModel: LlmService<*>

Or by name:

@Autowired
private lateinit var cheapest: LlmService<*>

Configuration:

Model: "gpt-4o-mini"
Provider: "OpenAI"
Mocked backend (no real API calls)

Best LLM Service

Pre-configured fake LLM service simulating gpt-4o.

@Autowired
@Qualifier("best")
private lateinit var bestModel: LlmService<*>

Or by name:

@Autowired
private lateinit var best: LlmService<*>

Configuration:

Model: "gpt-4o"
Provider: "OpenAI"
Mocked backend (no real API calls)

Embedding Service

Pre-configured fake embedding service with 1536 dimensions.

@Autowired
private lateinit var embeddingService: EmbeddingService

Configuration:

Model: "text-embedding-ada-002"
Provider: "OpenAI"
Uses FakeEmbeddingModel
Dimensions: 1536

Complete Examples

Example 1: Basic Test Setup

import com.embabel.common.test.ai.config.FakeAiConfiguration
import org.springframework.boot.test.context.SpringBootTest
import org.springframework.context.annotation.Import
import org.springframework.beans.factory.annotation.Autowired
import org.junit.jupiter.api.Test

@SpringBootTest
@Import(FakeAiConfiguration::class)
class MyServiceTest {

    @Autowired
    private lateinit var cheapest: LlmService<*>

    @Test
    fun `test service with fake LLM`() {
        val result = myService.process("input", cheapest)

        assertNotNull(result)
        // No API calls were made
    }
}

Example 2: Testing with Both Model Tiers

@SpringBootTest
@Import(FakeAiConfiguration::class)
class ModelTierTest {

    @Autowired
    @Qualifier("cheapest")
    private lateinit var cheapModel: LlmService<*>

    @Autowired
    @Qualifier("best")
    private lateinit var bestModel: LlmService<*>

    @Test
    fun `feature works with both tiers`() {
        // Test with cheap model
        val cheapResult = feature.execute(input, cheapModel)
        assertNotNull(cheapResult)

        // Test with best model
        val bestResult = feature.execute(input, bestModel)
        assertNotNull(bestResult)

        // Both succeed without API costs
    }
}

Example 3: Testing with Embeddings

@SpringBootTest
@Import(FakeAiConfiguration::class)
class EmbeddingTest {

    @Autowired
    private lateinit var embeddingService: EmbeddingService

    @Test
    fun `test with embeddings`() {
        val embedding = embeddingService.embed("test text")

        assertNotNull(embedding)
        assertEquals(1536, embedding.size)
    }
}

Example 4: All Services Together

@SpringBootTest
@Import(FakeAiConfiguration::class)
class CompleteTest {

    @Autowired
    private lateinit var cheapest: LlmService<*>

    @Autowired
    private lateinit var best: LlmService<*>

    @Autowired
    private lateinit var embeddingService: EmbeddingService

    @Test
    fun `test complete workflow`() {
        // Use cheap model for simple tasks
        val summary = summarizer.summarize(doc, cheapest)

        // Use best model for complex tasks
        val analysis = analyzer.analyze(data, best)

        // Use embeddings for search
        val embedding = embeddingService.embed(summary)

        assertNotNull(summary)
        assertNotNull(analysis)
        assertEquals(1536, embedding.size)
    }
}

Java Examples

Basic Java Setup

import com.embabel.common.test.ai.config.FakeAiConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.context.annotation.Import;
import org.springframework.beans.factory.annotation.Autowired;
import org.junit.jupiter.api.Test;

@SpringBootTest
@Import(FakeAiConfiguration.class)
public class MyServiceTest {

    @Autowired
    private LlmService<?> cheapest;

    @Test
    void testServiceWithFakeLlm() {
        String result = myService.process("input", cheapest);

        assertNotNull(result);
    }
}

Java with Qualifiers

@SpringBootTest
@Import(FakeAiConfiguration.class)
public class ModelTierTest {

    @Autowired
    @Qualifier("cheapest")
    private LlmService<?> cheapModel;

    @Autowired
    @Qualifier("best")
    private LlmService<?> bestModel;

    @Test
    void testBothTiers() {
        String cheapResult = feature.execute(input, cheapModel);
        String bestResult = feature.execute(input, bestModel);

        assertNotNull(cheapResult);
        assertNotNull(bestResult);
    }
}

Advanced Configuration

Overriding Beans

Replace specific beans with custom implementations:

@SpringBootTest
@Import(FakeAiConfiguration::class)
class CustomBeanTest {

    @TestConfiguration
    class CustomConfig {
        @Bean
        @Primary
        fun customCheapest(): LlmService<*> {
            // Custom mock with specific behavior
            val mockService = mockk<LlmService<*>>()
            every { mockService.generate(any<String>()) } returns "custom response"
            return mockService
        }
    }

    @Autowired
    private lateinit var cheapest: LlmService<*>

    @Test
    fun `test with custom bean`() {
        val result = cheapest.generate("test")
        assertEquals("custom response", result)
    }
}

Custom Embedding Dimensions

Override embedding service with different dimensions:

@SpringBootTest
class CustomEmbeddingTest {

    @TestConfiguration
    class TestConfig {
        @Bean
        @Primary
        fun customEmbedding(): EmbeddingService {
            val fakeModel = FakeEmbeddingModel(dimensions = 768)
            return SpringAiEmbeddingService(
                fakeModel,
                "custom-model",
                "CustomProvider"
            )
        }
    }

    @Autowired
    private lateinit var embeddingService: EmbeddingService

    @Test
    fun `test with 768 dimensions`() {
        val embedding = embeddingService.embed("test")
        assertEquals(768, embedding.size)
    }
}

Partial Configuration

Use only specific beans from FakeAiConfiguration:

@SpringBootTest
class PartialConfigTest {

    @TestConfiguration
    class TestConfig {
        @Bean
        fun embeddingService(): EmbeddingService {
            val fakeModel = FakeEmbeddingModel()
            return SpringAiEmbeddingService(
                fakeModel,
                "test-embedding",
                "TestProvider"
            )
        }

        // Don't import FakeAiConfiguration
        // Only use custom embedding service
    }

    @Autowired
    private lateinit var embeddingService: EmbeddingService

    @Test
    fun `test with only embedding service`() {
        val embedding = embeddingService.embed("test")
        assertNotNull(embedding)
    }
}

Integration with Other Test Configurations

With MockitoIntegrationTest

Combine with Mockito integration testing:

@SpringBootTest
@Import(FakeAiConfiguration::class)
class CombinedTest : EmbabelMockitoIntegrationTest() {

    @Autowired
    private lateinit var embeddingService: EmbeddingService

    @Test
    fun `test with both stubbing and fake beans`() {
        // Stub LLM operations
        whenGenerateText { it.contains("test") }
            .thenReturn("stubbed response")

        // Use fake embedding service
        val embedding = embeddingService.embed("test")

        // Execute code
        val result = myAgent.process("test")

        // Verify
        verifyGenerateText { it.contains("test") }
        assertNotNull(result)
        assertEquals(1536, embedding.size)
    }
}

With Test Slices

Use with Spring Boot test slices:

@DataJpaTest
@Import(FakeAiConfiguration::class)
class RepositoryTest {

    @Autowired
    private lateinit var embeddingService: EmbeddingService

    @Autowired
    private lateinit var repository: DocumentRepository

    @Test
    fun `test repository with embeddings`() {
        val embedding = embeddingService.embed("test")

        val document = Document("test", embedding)
        repository.save(document)

        val found = repository.findById(document.id)
        assertNotNull(found)
    }
}

Testing Patterns

Pattern 1: Strategy Testing

Test code works with different model strategies:

@Test
fun `test model strategy selection`() {
    // Simple task - use cheap model
    val simpleResult = taskExecutor.execute(
        simpleTask,
        strategy = ModelStrategy.CHEAP
    )

    // Complex task - use best model
    val complexResult = taskExecutor.execute(
        complexTask,
        strategy = ModelStrategy.BEST
    )

    assertNotNull(simpleResult)
    assertNotNull(complexResult)
}

Pattern 2: Fallback Testing

Test fallback from best to cheap:

@Test
fun `test fallback to cheaper model`() {
    val processor = SmartProcessor(best, cheapest)

    // Configure best to fail
    every { best.generate(any()) } throws RuntimeException()

    // Should fallback to cheapest
    val result = processor.processWithFallback(input)

    assertNotNull(result)
}

Pattern 3: Cost-Aware Testing

Test cost-optimization logic:

@Test
fun `test cost optimization`() {
    val optimizer = CostOptimizer(cheapest, best)

    // Small input - should use cheap
    val cheapResult = optimizer.optimize(smallInput)
    verify(exactly = 1) { cheapest.generate(any()) }

    // Large input - should use best
    val bestResult = optimizer.optimize(largeInput)
    verify(exactly = 1) { best.generate(any()) }
}

Troubleshooting

Bean Not Found

Problem: Spring cannot find the fake AI beans.

Solution: Ensure @Import(FakeAiConfiguration::class) is present:

@SpringBootTest
@Import(FakeAiConfiguration::class)  // Must have this
class MyTest { ... }

Wrong Bean Injected

Problem: Unexpected bean is injected.

Solution: Use qualifiers explicitly:

@Autowired
@Qualifier("cheapest")  // Be explicit
private lateinit var cheapModel: LlmService<*>

Configuration Conflicts

Problem: Multiple configurations conflict.

Solution: Use @Primary for custom beans:

@Bean
@Primary  // This takes precedence
fun customService(): LlmService<*> { ... }

Type Mismatches

Problem: Generic type issues with LlmService.

Solution: Use wildcard:

// Correct
private lateinit var llmService: LlmService<*>

// Or specific type if known
private lateinit var llmService: LlmService<OpenAiChatOptions>

Best Practices

Always Import Configuration: Don't forget @Import(FakeAiConfiguration::class)
Use Qualifiers When Needed: Be explicit about which bean you want
Override for Custom Behavior: Use @Primary when overriding beans
Test Multiple Tiers: Verify code works with both cheap and best models
Combine with Mockito: Use both fake beans and Mockito stubs together
Keep Tests Fast: Fake services are instant - take advantage of this

Next Steps

Testing Embeddings Guide - Test embedding operations
Testing Options Converters Guide - Test converters
Test Configuration API - Complete API reference
Test Configuration Module - Module details

tessl/maven-com-embabel-agent--embabel-agent-test-support

spring-test-setup.mddocs/guides/

Spring Test Setup Guide

What is FakeAiConfiguration?

Prerequisites

Basic Setup Pattern

Step 1: Add Configuration Import

Step 2: Inject Beans

Step 3: Use in Tests

Available Beans

Cheapest LLM Service

Best LLM Service

Embedding Service

Complete Examples

Example 1: Basic Test Setup

Example 2: Testing with Both Model Tiers

Example 3: Testing with Embeddings

Example 4: All Services Together

Java Examples

Basic Java Setup

Java with Qualifiers

Advanced Configuration

Overriding Beans

Custom Embedding Dimensions

Partial Configuration

Integration with Other Test Configurations

With MockitoIntegrationTest

With Test Slices

Testing Patterns

Pattern 1: Strategy Testing

Pattern 2: Fallback Testing

Pattern 3: Cost-Aware Testing

Troubleshooting

Bean Not Found

Wrong Bean Injected

Configuration Conflicts

Type Mismatches

Best Practices

Next Steps

Related Topics

tessl/maven-com-embabel-agent--embabel-agent-test-support

spring-test-setup.md.css-3qkkll{font-size:var(--chakra-font-sizes-sm);font-weight:var(--chakra-font-weights-normal);color:var(--chakra-colors-gray-300);}docs/guides/

Spring Test Setup Guide

What is FakeAiConfiguration?

Prerequisites

Basic Setup Pattern

Step 1: Add Configuration Import

Step 2: Inject Beans

Step 3: Use in Tests

Available Beans

Cheapest LLM Service

Best LLM Service

Embedding Service

Complete Examples

Example 1: Basic Test Setup

Example 2: Testing with Both Model Tiers

Example 3: Testing with Embeddings

Example 4: All Services Together

Java Examples

Basic Java Setup

Java with Qualifiers

Advanced Configuration

Overriding Beans

Custom Embedding Dimensions

Partial Configuration

Integration with Other Test Configurations

With MockitoIntegrationTest

With Test Slices

Testing Patterns

Pattern 1: Strategy Testing

Pattern 2: Fallback Testing

Pattern 3: Cost-Aware Testing

Troubleshooting

Bean Not Found

Wrong Bean Injected

Configuration Conflicts

Type Mismatches

Best Practices

Next Steps

Related Topics

spring-test-setup.mddocs/guides/