Embabel Agent Test Support

A comprehensive multi-module test support framework for Embabel Agent applications. Test AI agent applications without requiring actual LLM API calls or API keys.

Quick Navigation

New to this framework? Start here:

• Installation - Add dependencies to your project
• Your First Test - Write your first test in 2 minutes
• Common Tasks - Top tasks you'll perform

Need to use a specific feature?

• Stubbing API - Mock LLM responses
• Verification API - Assert LLM was called correctly
• FakeEmbeddingModel API - Test embeddings without API calls
• Test Configuration API - Spring test setup

Working on a specific task?

• Stubbing LLM Calls - How to stub LLM operations
• Verifying LLM Usage - How to verify LLM interactions
• Testing Embeddings - How to test embedding operations
• Spring Test Setup - How to configure Spring tests
• Testing Options Converters - How to test model options

Want to understand the modules?

• Integration Testing Module - embabel-agent-test
• Fake AI Services Module - embabel-agent-test-common
• Test Configuration Module - embabel-agent-test-internal

Looking for advanced patterns?

• Testing Patterns - Common testing patterns and recipes

Need reference information?

• Types Reference - All type definitions
• Architecture - How modules relate to each other

Overview

The Embabel Agent Test Support framework enables comprehensive testing of AI agent applications without requiring actual LLM API calls or API keys. It provides three complementary modules that work independently or together based on your testing needs.

What This Framework Provides

• No API Keys Required: Test agent applications without OpenAI or other LLM API keys
• No Network Calls: All AI operations are mocked or faked locally
• Fast Test Execution: Tests run instantly without waiting for API responses
• Comprehensive Stubbing: Fine-grained control over LLM behavior in tests
• Detailed Verification: Assert exact LLM interactions and parameters
• Spring Boot Integration: Seamless integration with Spring Boot test infrastructure
• Multi-Language: Java and Kotlin support throughout
• Flexible Configuration: Mix and match modules based on testing needs

The Three Modules

embabel-agent-test-support (parent POM)
├── embabel-agent-test              → Mockito integration testing
├── embabel-agent-test-common       → Fake AI services
└── embabel-agent-test-internal     → Spring test configuration

1. embabel-agent-test - Integration Testing

Mockito-based integration testing for agent applications.

Use when: Writing integration tests that need to mock LLM operations and verify agent behavior.

Key capability: Stub and verify LLM text generation and object creation.

Learn more: Integration Testing Module

2. embabel-agent-test-common - Fake AI Services

Fake implementations of AI services for testing without API calls.

Use when: Testing embedding operations, vector storage, or semantic search without real AI services.

Key capability: Generate fake embeddings with configurable dimensions.

Learn more: Fake AI Services Module

3. embabel-agent-test-internal - Test Configuration

Spring test configuration and testing utilities.

Use when: Setting up Spring Boot tests with complete fake AI infrastructure or testing custom model configurations.

Key capability: Provides pre-configured fake LLM and embedding service beans.

Learn more: Test Configuration Module

Package Information

• Package Type: Maven (parent POM with 3 modules)
• Language: Java and Kotlin
• Group ID: com.embabel.agent
• Artifact ID: embabel-agent-test-support
• Version: 0.3.3

See Installation for dependency configuration.

Quick Start Example

Here's a complete integration test using all three modules:

import com.embabel.agent.test.integration.EmbabelMockitoIntegrationTest;
import com.embabel.common.test.ai.config.FakeAiConfiguration;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.context.annotation.Import;
import org.springframework.beans.factory.annotation.Autowired;
import org.junit.jupiter.api.Test;
import static org.junit.jupiter.api.Assertions.*;

@SpringBootTest
@Import(FakeAiConfiguration.class)
public class MyAgentTest extends EmbabelMockitoIntegrationTest {

    @Autowired
    private LlmService<?> cheapest;

    @Autowired
    private EmbeddingService embeddingService;

    @Test
    void testCompleteAgentFlow() {
        // Stub LLM operations
        whenGenerateText(prompt -> prompt.contains("analyze"))
            .thenReturn("Analysis complete");

        // Execute agent with fake services
        var result = agentPlatform.executeFlow(input, cheapest, embeddingService);

        // Verify interactions
        verifyGenerateText(prompt -> prompt.contains("analyze"));
        assertNotNull(result);
    }
}

Next steps: Your First Test

Core Concepts

Stubbing

Stubbing allows you to control what the LLM returns in your tests:

// Stub text generation
whenGenerateText(prompt -> prompt.contains("hello"))
    .thenReturn("Hello, world!");

// Stub object creation
whenCreateObject(prompt -> prompt.contains("extract"), Person.class)
    .thenReturn(new Person("John"));

Learn more: Stubbing API | Stubbing Guide

Verification

Verification allows you to assert that the LLM was called with expected inputs:

// Verify text generation occurred
verifyGenerateText(prompt -> prompt.contains("expected"));

// Verify object creation occurred
verifyCreateObject(prompt -> prompt.contains("extract"), Person.class);

Learn more: Verification API | Verification Guide

Fake Services

Fake services provide test implementations that work without API calls:

// Create fake embedding model
val embeddingModel = FakeEmbeddingModel(dimensions = 1536)

// Generate test embeddings
val embedding = embeddingModel.embed(Document("test content"))
// Returns random 1536-dimensional embedding

Learn more: FakeEmbeddingModel API | Testing Embeddings Guide

Test Configuration

Spring test configuration provides pre-configured fake beans:

@SpringBootTest
@Import(FakeAiConfiguration::class)
class MyTest {
    @Autowired
    private lateinit var cheapest: LlmService<*>  // Fake gpt-4o-mini

    @Autowired
    private lateinit var best: LlmService<*>      // Fake gpt-4o

    @Autowired
    private lateinit var embeddingService: EmbeddingService  // Fake embeddings
}

Learn more: Test Configuration API | Spring Test Setup Guide

Common Use Cases

Testing Agent Text Generation

@Test
void testAgentGeneratesText() {
    whenGenerateText(p -> p.contains("summarize"))
        .thenReturn("This is a summary");

    String result = myAgent.summarize(document);

    verifyGenerateText(p -> p.contains("summarize"));
    assertEquals("This is a summary", result);
}

Testing Agent Object Creation

@Test
void testAgentExtractsData() {
    Person expected = new Person("Alice", 30);
    whenCreateObject(p -> p.contains("extract"), Person.class)
        .thenReturn(expected);

    Person result = myAgent.extractPerson(text);

    verifyCreateObject(p -> p.contains("extract"), Person.class);
    assertEquals(expected, result);
}

Testing Embedding Operations

@Test
fun `test semantic search without API`() {
    val embeddingModel = FakeEmbeddingModel(dimensions = 768)
    val searchEngine = SemanticSearchEngine(embeddingModel)

    searchEngine.indexDocuments(listOf("doc1", "doc2", "doc3"))

    val results = searchEngine.search("query", topK = 2)
    assertEquals(2, results.size)
}

Testing with Different Model Tiers

@Test
fun `test feature works with all model tiers`() {
    val cheapResult = feature.execute(input, cheapModel)
    val bestResult = feature.execute(input, bestModel)

    // Both should succeed
    assertNotNull(cheapResult)
    assertNotNull(bestResult)
}

More examples: Common Tasks | Testing Patterns

Module Dependencies

Understanding how the modules relate:

• embabel-agent-test: Depends on embabel-agent-api, provides Mockito integration
• embabel-agent-test-common: Depends on embabel-agent-common and Spring AI, provides fake models
• embabel-agent-test-internal: Depends on both test modules plus embabel-agent-domain, provides test configuration

All modules are designed to work independently or together based on your testing needs.

Learn more: Architecture Reference

Key Features Summary

Getting Help

I want to...

• Get started quickly → Your First Test
• Stub an LLM call → Stubbing Guide
• Verify LLM was called → Verification Guide
• Test embeddings → Testing Embeddings Guide
• Set up Spring tests → Spring Test Setup Guide
• Understand a module → Modules
• See example patterns → Testing Patterns
• Look up API signatures → API Reference
• Understand types → Types Reference

Version Information

• Current Version: 0.3.3
• Java Version: Compatible with Java 11+
• Kotlin Version: Compatible with Kotlin 1.9+
• Spring Boot: Compatible with Spring Boot 3.x
• Spring AI: Uses Spring AI framework for interfaces