tessl/maven-com-embabel-agent--embabel-agent-test-support
Multi-module test support framework for Embabel Agent applications providing integration testing, mock AI services, and test configuration utilities
common-tasks.mddocs/quick-start/
Common Tasks
Quick reference for the most common testing tasks.
Task 1: Stub LLM Text Generation
Control what text the LLM returns in your tests.
whenGenerateText(prompt -> prompt.contains("keyword"))
.thenReturn("Mocked response");Example:
@Test
void testTextGeneration() {
whenGenerateText(p -> p.contains("summarize"))
.thenReturn("This is a summary");
String result = myAgent.summarize(document);
assertEquals("This is a summary", result);
}Learn more: Stubbing Guide | Stubbing API
Task 2: Stub LLM Object Creation
Control what structured object the LLM returns.
whenCreateObject(prompt -> prompt.contains("extract"), MyClass.class)
.thenReturn(expectedObject);Example:
@Test
void testObjectExtraction() {
Person person = new Person("Alice", 30);
whenCreateObject(p -> p.contains("extract"), Person.class)
.thenReturn(person);
Person result = myAgent.extractPerson(text);
assertEquals(person, result);
}Learn more: Stubbing Guide | Stubbing API
Task 3: Verify LLM Was Called
Assert that your code called the LLM with expected prompt.
verifyGenerateText(prompt -> prompt.contains("expected"));Example:
@Test
void testLlmWasCalled() {
myAgent.process("user input");
verifyGenerateText(p ->
p.contains("user input") && p.contains("process")
);
}Learn more: Verification Guide | Verification API
Task 4: Verify LLM Was NOT Called
Assert that your code did not make LLM calls.
verifyNoInteractions();Example:
@Test
void testCachedPath() {
// Execute cached code path
myAgent.getCachedResult();
// Verify no LLM calls were made
verifyNoInteractions();
}Learn more: Verification Guide
Task 5: Test Embeddings Without API
Generate fake embeddings for testing.
val embeddingModel = FakeEmbeddingModel(dimensions = 1536)
val embedding = embeddingModel.embed(Document("test content"))Example:
@Test
fun `test semantic search`() {
val model = FakeEmbeddingModel(dimensions = 768)
val searchEngine = SemanticSearchEngine(model)
searchEngine.indexDocuments(listOf("doc1", "doc2"))
val results = searchEngine.search("query")
assertTrue(results.isNotEmpty())
}Learn more: Testing Embeddings Guide | FakeEmbeddingModel API
Task 6: Set Up Spring Tests with Fake AI
Configure Spring Boot tests with pre-configured fake LLM services.
@SpringBootTest
@Import(FakeAiConfiguration::class)
class MyTest {
@Autowired
private lateinit var cheapest: LlmService<*>
@Autowired
private lateinit var best: LlmService<*>
}Example:
@SpringBootTest
@Import(FakeAiConfiguration::class)
class MyServiceTest {
@Autowired
private lateinit var cheapest: LlmService<*>
@Test
fun `test with fake LLM`() {
val result = myService.process(input, cheapest)
assertNotNull(result)
}
}Learn more: Spring Test Setup Guide | Test Configuration API
Task 7: Stub Multiple LLM Calls
Handle tests where your code makes multiple LLM calls.
@Test
void testMultipleCalls() {
// Stub first call
whenGenerateText(p -> p.contains("step1"))
.thenReturn("Result 1");
// Stub second call
whenGenerateText(p -> p.contains("step2"))
.thenReturn("Result 2");
// Execute
myAgent.multiStepProcess();
// Verify both
verifyGenerateText(p -> p.contains("step1"));
verifyGenerateText(p -> p.contains("step2"));
}Learn more: Testing Patterns
Task 8: Test with Different Model Tiers
Test that your code works with both cheap and expensive models.
@SpringBootTest
@Import(FakeAiConfiguration::class)
class ModelTierTest {
@Autowired
@Qualifier("cheapest")
private lateinit var cheapModel: LlmService<*>
@Autowired
@Qualifier("best")
private lateinit var bestModel: LlmService<*>
@Test
fun `test works with all tiers`() {
val cheapResult = feature.execute(input, cheapModel)
val bestResult = feature.execute(input, bestModel)
assertNotNull(cheapResult)
assertNotNull(bestResult)
}
}Learn more: Spring Test Setup Guide
Task 9: Capture and Inspect LLM Interactions
Capture actual prompt and interaction details for detailed assertions.
ArgumentCaptor<LlmInteraction> captor = captureLlmInteraction();
// ... perform verification ...
LlmInteraction interaction = captor.getValue();Example:
@Test
void testPromptDetails() {
whenGenerateText(p -> true).thenReturn("result");
myAgent.process("input");
ArgumentCaptor<LlmInteraction> captor = captureLlmInteraction();
verifyGenerateText(p -> true);
LlmInteraction interaction = captor.getValue();
assertEquals("gpt-4", interaction.getModel());
assertEquals(0.7, interaction.getTemperature());
}Learn more: Verification API
Task 10: Test Options Converter
Test that your custom options converter preserves core LLM values.
checkOptionsConverterPreservesCoreValues(optionsConverter)Example:
@Test
fun `test custom converter`() {
val converter = MyCustomOptionsConverter()
checkOptionsConverterPreservesCoreValues(converter)
}Learn more: Testing Options Converters Guide
Quick Reference Table
| Task | Method | Module |
|---|---|---|
| Stub text generation | whenGenerateText() | embabel-agent-test |
| Stub object creation | whenCreateObject() | embabel-agent-test |
| Verify text generation | verifyGenerateText() | embabel-agent-test |
| Verify object creation | verifyCreateObject() | embabel-agent-test |
| Verify no LLM calls | verifyNoInteractions() | embabel-agent-test |
| Fake embeddings | FakeEmbeddingModel() | embabel-agent-test-common |
| Spring test setup | @Import(FakeAiConfiguration) | embabel-agent-test-internal |
| Capture interactions | captureLlmInteraction() | embabel-agent-test |
| Test options converter | checkOptionsConverterPreservesCoreValues() | embabel-agent-test-internal |
Next Steps
- Stubbing Guide - Detailed stubbing techniques
- Verification Guide - Detailed verification techniques
- Testing Patterns - Common testing patterns
- API Reference - Complete API documentation