CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/maven-org-springframework-ai--spring-ai-spring-boot-autoconfigure

Spring AI Spring Boot Auto Configuration modules providing automatic setup for AI models, vector stores, MCP, and retry capabilities

Overview
Eval results
Files

edge-cases.mddocs/examples/

Edge Cases and Troubleshooting

Advanced scenarios, edge cases, and solutions for common issues.

Edge Case 1: Handling Large Context Windows

When working with models that support large context windows (100K+ tokens).

Problem

Standard chunking strategies may not be optimal for models with large context windows.

Solution

@Service
public class LargeContextHandler {
    private final ChatModel chatModel;
    private final int maxContextTokens;
    
    public LargeContextHandler(ChatModel chatModel) {
        this.chatModel = chatModel;
        this.maxContextTokens = 100000; // For Claude or GPT-4 Turbo
    }
    
    public String processLargeDocument(String document) {
        // For large context models, send entire document instead of chunking
        if (estimateTokens(document) <= maxContextTokens) {
            return chatModel.call(
                "Analyze this entire document: " + document
            );
        }
        
        // Fallback to chunking if too large
        return processInChunks(document);
    }
    
    private int estimateTokens(String text) {
        // Rough estimate: 1 token ≈ 4 characters
        return text.length() / 4;
    }
}

Edge Case 2: Rate Limit Exhaustion

When hitting rate limits across all retry attempts.

Problem

Rate limits exhausted, causing service disruption.

Solution

@Service
public class RateLimitedChatService {
    private final ChatModel chatModel;
    private final RateLimiter rateLimiter;
    private final Queue<CompletableFuture<String>> requestQueue;
    
    public RateLimitedChatService(ChatModel chatModel) {
        this.chatModel = chatModel;
        this.rateLimiter = RateLimiter.create(10.0); // 10 requests/second
        this.requestQueue = new ConcurrentLinkedQueue<>();
        startQueueProcessor();
    }
    
    public CompletableFuture<String> chat(String message) {
        CompletableFuture<String> future = new CompletableFuture<>();
        requestQueue.offer(future);
        
        // Process request when rate limit allows
        processNextRequest(message, future);
        
        return future;
    }
    
    private void processNextRequest(
            String message,
            CompletableFuture<String> future) {
        
        rateLimiter.acquire(); // Block until rate limit allows
        
        try {
            String response = chatModel.call(message);
            future.complete(response);
        } catch (Exception e) {
            future.completeExceptionally(e);
        }
    }
    
    private void startQueueProcessor() {
        // Background thread to process queue
        Executors.newSingleThreadExecutor().submit(() -> {
            while (true) {
                CompletableFuture<String> future = requestQueue.poll();
                if (future != null) {
                    // Process queued request
                }
                Thread.sleep(100);
            }
        });
    }
}

Edge Case 3: Token Limit Exceeded Mid-Conversation

When conversation history exceeds model token limits.

Problem

Conversation history grows too large, causing token limit errors.

Solution

@Service
public class TokenAwareChatService {
    private final ChatClient chatClient;
    private final ChatMemory chatMemory;
    private final int maxTokens = 8000;
    
    public String chat(String sessionId, String message) {
        List<Message> history = chatMemory.get(sessionId);
        
        // Trim history if too long
        List<Message> trimmedHistory = trimToTokenLimit(
            history,
            maxTokens - estimateTokens(message) - 1000 // Reserve for response
        );
        
        // Build prompt with trimmed history
        String response = chatClient.prompt()
            .user(message)
            .messages(trimmedHistory)
            .call()
            .content();
        
        chatMemory.add(sessionId, new UserMessage(message));
        chatMemory.add(sessionId, new AssistantMessage(response));
        
        return response;
    }
    
    private List<Message> trimToTokenLimit(
            List<Message> messages,
            int limit) {
        
        List<Message> trimmed = new ArrayList<>();
        int tokens = 0;
        
        // Keep most recent messages within limit
        for (int i = messages.size() - 1; i >= 0; i--) {
            Message msg = messages.get(i);
            int msgTokens = estimateTokens(msg.getContent());
            
            if (tokens + msgTokens > limit) {
                break;
            }
            
            trimmed.add(0, msg);
            tokens += msgTokens;
        }
        
        return trimmed;
    }
    
    private int estimateTokens(String text) {
        return text.length() / 4;
    }
}

Edge Case 4: Provider API Changes

When AI provider changes API without notice.

Problem

Provider API returns unexpected format, causing parsing errors.

Solution

@Service
public class ResilientChatService {
    private final ChatModel chatModel;
    
    public String chat(String message) {
        try {
            return chatModel.call(message);
        } catch (JsonProcessingException e) {
            log.error("API response format changed: {}", e.getMessage());
            
            // Fallback: Return raw response or error message
            return handleApiFormatChange(message, e);
        } catch (Exception e) {
            log.error("Unexpected error: {}", e.getMessage());
            throw e;
        }
    }
    
    private String handleApiFormatChange(
            String message,
            Exception e) {
        
        // Log for investigation
        log.error("API format issue", e);
        
        // Notify monitoring system
        alerting.sendAlert("API format changed", e);
        
        // Return graceful error
        return "I'm experiencing technical difficulties. " +
               "Please try again later.";
    }
}

Edge Case 5: Embedding Dimension Mismatch

When switching embedding models with different dimensions.

Problem

Existing vectors have different dimensions than new embedding model.

Solution

@Service
public class EmbeddingMigrationService {
    private final EmbeddingModel oldModel;
    private final EmbeddingModel newModel;
    private final VectorStore vectorStore;
    
    public void migrateEmbeddings() {
        // 1. Get all documents
        List<Document> allDocs = vectorStore.similaritySearch(
            SearchRequest.query("").withTopK(Integer.MAX_VALUE)
        );
        
        // 2. Delete old vectors
        vectorStore.delete(
            allDocs.stream()
                .map(Document::getId)
                .collect(Collectors.toList())
        );
        
        // 3. Re-embed with new model
        List<String> texts = allDocs.stream()
            .map(Document::getContent)
            .collect(Collectors.toList());
        
        EmbeddingResponse newEmbeddings = newModel.embedForResponse(texts);
        
        // 4. Create new documents with new embeddings
        List<Document> newDocs = new ArrayList<>();
        for (int i = 0; i < allDocs.size(); i++) {
            Document oldDoc = allDocs.get(i);
            Document newDoc = new Document(oldDoc.getContent());
            newDoc.getMetadata().putAll(oldDoc.getMetadata());
            newDoc.setEmbedding(
                newEmbeddings.getResults().get(i).getOutput()
            );
            newDocs.add(newDoc);
        }
        
        // 5. Add back to vector store
        vectorStore.add(newDocs);
    }
}

Edge Case 6: Concurrent Vector Store Updates

When multiple services update vector store simultaneously.

Problem

Race conditions in vector store operations.

Solution

@Service
public class ConcurrentSafeVectorStore {
    private final VectorStore vectorStore;
    private final Lock lock = new ReentrantLock();
    
    public void safeAdd(List<Document> documents) {
        lock.lock();
        try {
            // Check for duplicates before adding
            List<Document> uniqueDocs = documents.stream()
                .filter(doc -> !exists(doc.getId()))
                .collect(Collectors.toList());
            
            if (!uniqueDocs.isEmpty()) {
                vectorStore.add(uniqueDocs);
            }
        } finally {
            lock.unlock();
        }
    }
    
    public void safeDelete(List<String> ids) {
        lock.lock();
        try {
            vectorStore.delete(ids);
        } finally {
            lock.unlock();
        }
    }
    
    private boolean exists(String id) {
        try {
            List<Document> results = vectorStore.similaritySearch(
                SearchRequest.query("")
                    .withFilterExpression(
                        Filter.expression("id == '" + id + "'")
                    )
                    .withTopK(1)
            );
            return !results.isEmpty();
        } catch (Exception e) {
            return false;
        }
    }
}

Edge Case 7: MCP Server Crashes

When MCP server process crashes during operation.

Problem

Stdio MCP server process terminates unexpectedly.

Solution

@Service
public class ResilientMcpClient {
    private final List<McpSyncClient> clients;
    private final Map<String, Process> processes;
    
    @Scheduled(fixedRate = 30000) // Check every 30 seconds
    public void healthCheck() {
        for (McpSyncClient client : clients) {
            try {
                client.ping();
            } catch (Exception e) {
                log.error("MCP client {} unhealthy, attempting restart", 
                         client.getName());
                restartClient(client);
            }
        }
    }
    
    private void restartClient(McpSyncClient client) {
        try {
            // Close existing client
            client.close();
            
            // Restart process (for stdio transport)
            Process oldProcess = processes.get(client.getName());
            if (oldProcess != null && oldProcess.isAlive()) {
                oldProcess.destroy();
            }
            
            // Recreate client (implementation depends on transport type)
            // ...
            
        } catch (Exception e) {
            log.error("Failed to restart MCP client", e);
        }
    }
}

Edge Case 8: Native Image Reflection Issues

When custom classes aren't accessible in native image.

Problem

Reflection errors in GraalVM native image for custom tool types.

Solution

@Configuration
public class NativeImageConfiguration {
    
    @Bean
    public RuntimeHintsRegistrar customRuntimeHints() {
        return (hints, classLoader) -> {
            // Register custom tool classes for reflection
            hints.reflection()
                .registerType(MyCustomTool.class, 
                    MemberCategory.INVOKE_PUBLIC_METHODS,
                    MemberCategory.INVOKE_PUBLIC_CONSTRUCTORS)
                .registerType(MyCustomToolRequest.class,
                    MemberCategory.INVOKE_PUBLIC_METHODS)
                .registerType(MyCustomToolResponse.class,
                    MemberCategory.INVOKE_PUBLIC_METHODS);
            
            // Register custom resources
            hints.resources()
                .registerPattern("custom-prompts/*.txt")
                .registerPattern("mcp-configs/*.json");
            
            // Register for serialization
            hints.serialization()
                .registerType(MyCustomTool.class)
                .registerType(MyCustomToolRequest.class)
                .registerType(MyCustomToolResponse.class);
        };
    }
}

Create META-INF/native-image/reflect-config.json:

[
  {
    "name": "com.example.MyCustomTool",
    "allDeclaredConstructors": true,
    "allPublicConstructors": true,
    "allDeclaredMethods": true,
    "allPublicMethods": true,
    "allDeclaredFields": true
  }
]

Edge Case 9: Memory Leaks in Long-Running Conversations

When conversation memory grows unbounded.

Problem

Memory usage increases indefinitely with long conversations.

Solution

@Service
public class BoundedChatMemory {
    private final ChatMemory chatMemory;
    private final int maxMessagesPerSession = 100;
    private final Duration sessionTtl = Duration.ofHours(24);
    private final Map<String, Instant> sessionLastAccess;
    
    public BoundedChatMemory(ChatMemory chatMemory) {
        this.chatMemory = chatMemory;
        this.sessionLastAccess = new ConcurrentHashMap<>();
        startCleanupTask();
    }
    
    public void add(String sessionId, Message message) {
        List<Message> history = chatMemory.get(sessionId);
        
        // Limit message count
        if (history.size() >= maxMessagesPerSession) {
            // Remove oldest messages
            int toRemove = history.size() - maxMessagesPerSession + 1;
            history = history.subList(toRemove, history.size());
            
            // Clear and re-add
            chatMemory.clear(sessionId);
            history.forEach(msg -> chatMemory.add(sessionId, msg));
        }
        
        chatMemory.add(sessionId, message);
        sessionLastAccess.put(sessionId, Instant.now());
    }
    
    @Scheduled(fixedRate = 3600000) // Every hour
    private void startCleanupTask() {
        Instant cutoff = Instant.now().minus(sessionTtl);
        
        sessionLastAccess.entrySet().removeIf(entry -> {
            if (entry.getValue().isBefore(cutoff)) {
                chatMemory.clear(entry.getKey());
                return true;
            }
            return false;
        });
    }
}

Edge Case 10: Vector Store Index Corruption

When vector store index becomes corrupted.

Problem

Vector store returns inconsistent or no results.

Solution

@Service
public class VectorStoreHealthCheck {
    private final VectorStore vectorStore;
    
    @Scheduled(cron = "0 0 2 * * *") // Daily at 2 AM
    public void healthCheck() {
        try {
            // Test basic operations
            Document testDoc = new Document("test");
            testDoc.setId("health-check-" + UUID.randomUUID());
            
            vectorStore.add(List.of(testDoc));
            
            List<Document> results = vectorStore.similaritySearch(
                SearchRequest.query("test").withTopK(1)
            );
            
            vectorStore.delete(List.of(testDoc.getId()));
            
            if (results.isEmpty()) {
                log.error("Vector store health check failed: no results");
                alerting.sendAlert("Vector store health check failed");
            }
            
        } catch (Exception e) {
            log.error("Vector store health check error", e);
            alerting.sendAlert("Vector store error: " + e.getMessage());
        }
    }
    
    public void rebuildIndex() {
        // Implementation depends on vector store type
        // For PGVector:
        executeSql("REINDEX INDEX vector_index");
        
        // For others, may need to recreate index
        // or rebuild from source data
    }
}

Common Configuration Issues

Issue: Beans Not Created

Symptom: Expected beans not available in context.

Diagnosis:

# Enable autoconfiguration debug
logging.level.org.springframework.boot.autoconfigure=DEBUG

Solution:

  1. Check conditional requirements are met
  2. Verify required classes on classpath
  3. Check property configuration

Issue: Provider Authentication Failures

Symptom: 401/403 errors despite correct API key.

Solutions:

# Verify API key format (no whitespace)
spring.ai.openai.api-key=${OPENAI_API_KEY:}

# Check for provider-specific requirements
spring.ai.azure.openai.endpoint=https://RESOURCE.openai.azure.com
spring.ai.azure.openai.api-key=KEY

# Enable request logging
logging.level.org.springframework.web.client.RestTemplate=DEBUG

Issue: Vector Store Schema Initialization Fails

Symptom: Schema creation errors on startup.

Solutions:

# Check database permissions
spring.ai.vectorstore.pgvector.initialize-schema=always

# Use platform-specific SQL
spring.ai.vectorstore.jdbc.platform=postgresql
spring.ai.vectorstore.jdbc.schema=classpath:custom-schema-postgresql.sql

# Disable if using migration tools
spring.ai.vectorstore.pgvector.initialize-schema=never

Performance Issues

Slow Embedding Generation

Problem: Embedding generation takes too long.

Solution:

@Service
public class BatchEmbeddingService {
    private final EmbeddingModel embeddingModel;
    
    public List<List<Double>> embedBatch(List<String> texts) {
        // Batch embed instead of one-by-one
        EmbeddingResponse response = embeddingModel.embedForResponse(texts);
        
        return response.getResults().stream()
            .map(result -> result.getOutput())
            .collect(Collectors.toList());
    }
}

Slow Vector Search

Problem: Vector search queries are slow.

Solutions:

# Use HNSW index for better performance
spring.ai.vectorstore.pgvector.index-type=HNSW

# Reduce dimensions if possible
spring.ai.vectorstore.milvus.embedding-dimension=768

# Use similarity threshold to limit results
# In code:
# SearchRequest.query(text)
#     .withTopK(10)
#     .withSimilarityThreshold(0.7)

Further Reading

  • Real-World Scenarios
  • Troubleshooting Reference
  • Configuration Properties

Install with Tessl CLI

npx tessl i tessl/maven-org-springframework-ai--spring-ai-spring-boot-autoconfigure

docs

examples

complete-applications.md

edge-cases.md

multi-provider.md

real-world-scenarios.md

index.md

tile.json