tessl/maven-org-springframework-ai--spring-ai-model

Core model interfaces and abstractions for Spring AI framework providing portable API for chat, embeddings, images, audio, and tool calling across multiple AI providers

Overview

Eval results

Files

Observability

Name: tessl/maven-org-springframework-ai--spring-ai-model
Author: tessl

Built-in observability integration with Micrometer for monitoring AI operations, including metrics, traces, and logs for chat, embeddings, images, audio, moderation, and tool executions.

Core Observability Types

ModelObservationContext

Base context class for all model observations.

public class ModelObservationContext<TReq extends ModelRequest<?>, TRes extends ModelResponse<?>> {
    /**
     * Get the model request.
     *
     * @return the request
     */
    public TReq getRequest();

    /**
     * Get the model response.
     *
     * @return the response
     */
    public TRes getResponse();

    /**
     * Set the model response.
     *
     * @param response the response
     */
    public void setResponse(TRes response);
}

ErrorLoggingObservationHandler

Logs errors from model operations.

public class ErrorLoggingObservationHandler
    implements ObservationHandler<ModelObservationContext<?, ?>> {
    /**
     * Handles logging of errors during model operations.
     */
}

ModelUsageMetricsGenerator

Generates usage metrics for model operations.

public class ModelUsageMetricsGenerator {
    /**
     * Generates metrics for tracking model usage including tokens and costs.
     */
}

Capabilities

Chat Model Observability

ChatModelObservationContext

Context for observing chat model operations.

public class ChatModelObservationContext extends ModelObservationContext<Prompt, ChatResponse> {
    /**
     * Construct a ChatModelObservationContext.
     *
     * @param request the chat prompt
     * @param chatModel the chat model being observed
     */
    public ChatModelObservationContext(Prompt request, ChatModel chatModel);

    /**
     * Set the chat response.
     *
     * @param response the response
     */
    public void setResponse(ChatResponse response);

    /**
     * Get the chat request.
     *
     * @return the prompt
     */
    public Prompt getRequest();

    /**
     * Get the chat response.
     *
     * @return the response
     */
    public ChatResponse getResponse();
}

ChatModelObservationConvention Interface

Convention for naming and tagging chat observations.

public interface ChatModelObservationConvention
    extends ObservationConvention<ChatModelObservationContext> {
    // Methods for defining observation names and key values
}

DefaultChatModelObservationConvention

Default implementation of chat model observation convention.

public class DefaultChatModelObservationConvention
    implements ChatModelObservationConvention {
    // Default naming and tagging for chat observations
}

ChatModelObservationHandler Implementations

public class ChatModelCompletionObservationHandler
    implements ObservationHandler<ChatModelObservationContext> {
    /**
     * Logs chat completions.
     */
}

public class ChatModelPromptContentObservationHandler
    implements ObservationHandler<ChatModelObservationContext> {
    /**
     * Logs prompt content for debugging.
     */
}

public class ChatModelMeterObservationHandler
    implements ObservationHandler<ChatModelObservationContext> {
    /**
     * Records metrics like token usage, duration, etc.
     */
}

ChatModelObservationDocumentation

public enum ChatModelObservationDocumentation implements ObservationDocumentation {
    /**
     * Chat model operation observation.
     */
    CHAT_MODEL_OPERATION;
}

Embedding Model Observability

EmbeddingModelObservationContext

Context for observing embedding model operations.

public class EmbeddingModelObservationContext
    extends ModelObservationContext<EmbeddingRequest, EmbeddingResponse> {
    /**
     * Construct an EmbeddingModelObservationContext.
     *
     * @param request the embedding request
     * @param embeddingModel the embedding model
     */
    public EmbeddingModelObservationContext(
        EmbeddingRequest request,
        EmbeddingModel embeddingModel
    );
}

EmbeddingModelObservationConvention Interface

Convention for embedding observations.

public interface EmbeddingModelObservationConvention
    extends ObservationConvention<EmbeddingModelObservationContext> {
    // Methods for embedding observation naming and tagging
}

DefaultEmbeddingModelObservationConvention

public class DefaultEmbeddingModelObservationConvention
    implements EmbeddingModelObservationConvention {
    // Default convention implementation
}

EmbeddingModelMeterObservationHandler

public class EmbeddingModelMeterObservationHandler
    implements ObservationHandler<EmbeddingModelObservationContext> {
    /**
     * Records embedding metrics.
     */
}

EmbeddingModelObservationDocumentation

public enum EmbeddingModelObservationDocumentation implements ObservationDocumentation {
    EMBEDDING_MODEL_OPERATION;
}

Image Model Observability

ImageModelObservationContext

Context for observing image model operations.

public class ImageModelObservationContext
    extends ModelObservationContext<ImagePrompt, ImageResponse> {
    /**
     * Construct an ImageModelObservationContext.
     *
     * @param request the image prompt
     * @param imageModel the image model
     */
    public ImageModelObservationContext(
        ImagePrompt request,
        ImageModel imageModel
    );
}

ImageModelObservationConvention Interface

public interface ImageModelObservationConvention
    extends ObservationConvention<ImageModelObservationContext> {
    // Methods for image observation naming and tagging
}

DefaultImageModelObservationConvention

public class DefaultImageModelObservationConvention
    implements ImageModelObservationConvention {
    // Default convention implementation
}

ImageModelPromptContentObservationHandler

public class ImageModelPromptContentObservationHandler
    implements ObservationHandler<ImageModelObservationContext> {
    /**
     * Logs image prompts.
     */
}

ImageModelObservationDocumentation

public enum ImageModelObservationDocumentation implements ObservationDocumentation {
    IMAGE_MODEL_OPERATION;
}

Observation Key Names

Observation documentation enums define metric and trace key names for each model type.

Chat Model Observation Keys

public enum ChatModelObservationDocumentation implements ObservationDocumentation {
    CHAT_MODEL_OPERATION;

    /**
     * Low cardinality keys for chat model observations.
     */
    public enum LowCardinalityKeyNames {
        AI_OPERATION_TYPE,
        AI_PROVIDER,
        REQUEST_MODEL,
        RESPONSE_MODEL
    }

    /**
     * High cardinality keys for chat model observations.
     */
    public enum HighCardinalityKeyNames {
        REQUEST_MESSAGES,
        REQUEST_TEMPERATURE,
        REQUEST_TOP_P,
        REQUEST_MAX_TOKENS,
        RESPONSE_ID,
        RESPONSE_FINISH_REASON,
        USAGE_INPUT_TOKENS,
        USAGE_OUTPUT_TOKENS,
        USAGE_TOTAL_TOKENS
    }
}

Embedding Model Observation Keys

public enum EmbeddingModelObservationDocumentation implements ObservationDocumentation {
    EMBEDDING_MODEL_OPERATION;

    /**
     * Low cardinality keys for embedding model observations.
     */
    public enum LowCardinalityKeyNames {
        AI_OPERATION_TYPE,
        AI_PROVIDER,
        REQUEST_MODEL,
        RESPONSE_MODEL
    }

    /**
     * High cardinality keys for embedding model observations.
     */
    public enum HighCardinalityKeyNames {
        REQUEST_EMBEDDING_DIMENSIONS,
        USAGE_INPUT_TOKENS,
        USAGE_TOTAL_TOKENS
    }
}

Image Model Observation Keys

public enum ImageModelObservationDocumentation implements ObservationDocumentation {
    IMAGE_MODEL_OPERATION;

    /**
     * Low cardinality keys for image model observations.
     */
    public enum LowCardinalityKeyNames {
        AI_OPERATION_TYPE,
        AI_PROVIDER,
        REQUEST_MODEL
    }

    /**
     * High cardinality keys for image model observations.
     */
    public enum HighCardinalityKeyNames {
        REQUEST_IMAGE_SIZE,
        REQUEST_IMAGE_STYLE,
        REQUEST_IMAGE_QUALITY,
        RESPONSE_IMAGE_COUNT
    }
}

Tool Calling Observation Keys

public enum ToolCallingObservationDocumentation implements ObservationDocumentation {
    TOOL_CALLING;

    /**
     * Low cardinality keys for tool calling observations.
     */
    public enum LowCardinalityKeyNames {
        TOOL_NAME,
        TOOL_TYPE
    }

    /**
     * High cardinality keys for tool calling observations.
     */
    public enum HighCardinalityKeyNames {
        TOOL_INPUT,
        TOOL_OUTPUT,
        TOOL_EXECUTION_TIME
    }
}

Tool Calling Observability

ToolCallingObservationContext

Context for observing tool executions.

public class ToolCallingObservationContext extends Observation.Context {
    /**
     * Construct a ToolCallingObservationContext.
     *
     * @param toolDefinition the tool definition
     * @param toolInput the JSON input to the tool
     */
    public ToolCallingObservationContext(
        ToolDefinition toolDefinition,
        String toolInput
    );

    /**
     * Set the tool result.
     *
     * @param result the tool result
     */
    public void setToolResult(String result);

    /**
     * Set the tool error.
     *
     * @param error the error that occurred
     */
    public void setError(Throwable error);

    /**
     * Get the tool definition.
     *
     * @return the tool definition
     */
    public ToolDefinition getToolDefinition();

    /**
     * Get the tool input.
     *
     * @return the input string
     */
    public String getToolInput();

    /**
     * Get the tool result.
     *
     * @return the result string
     */
    public String getToolResult();
}

ToolCallingObservationConvention Interface

public interface ToolCallingObservationConvention
    extends ObservationConvention<ToolCallingObservationContext> {
    // Methods for tool observation naming and tagging
}

DefaultToolCallingObservationConvention

public class DefaultToolCallingObservationConvention
    implements ToolCallingObservationConvention {
    // Default convention implementation
}

ToolCallingContentObservationFilter

public class ToolCallingContentObservationFilter implements ObservationFilter {
    /**
     * Filters tool content for logging.
     */
}

ToolCallingObservationDocumentation

public enum ToolCallingObservationDocumentation implements ObservationDocumentation {
    TOOL_CALLING_OPERATION;
}

Model Observability Base

ModelObservationContext

Base context for all model observations.

public class ModelObservationContext<REQ, RES> extends Observation.Context {
    /**
     * Get the request.
     *
     * @return the request
     */
    public REQ getRequest();

    /**
     * Set the request.
     *
     * @param request the request
     */
    public void setRequest(REQ request);

    /**
     * Get the response.
     *
     * @return the response
     */
    public RES getResponse();

    /**
     * Set the response.
     *
     * @param response the response
     */
    public void setResponse(RES response);

    // Additional setters for observation metadata
    public void setModelName(String modelName);
    public void setProvider(String provider);
    public void setRequestId(String requestId);
}

ErrorLoggingObservationHandler

public class ErrorLoggingObservationHandler implements ObservationHandler {
    /**
     * Logs errors that occur during observations.
     */
}

ModelUsageMetricsGenerator

public class ModelUsageMetricsGenerator {
    /**
     * Generates usage metrics from model responses.
     * Static utility methods for extracting metrics.
     */
}

Usage Examples

Enabling Chat Model Observability

import io.micrometer.observation.ObservationRegistry;
import org.springframework.ai.chat.observation.*;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class ObservabilityConfig {

    @Bean
    public ObservationRegistry observationRegistry() {
        ObservationRegistry registry = ObservationRegistry.create();

        // Register handlers
        registry.observationConfig()
            .observationHandler(new ChatModelCompletionObservationHandler())
            .observationHandler(new ChatModelPromptContentObservationHandler())
            .observationHandler(new ChatModelMeterObservationHandler());

        return registry;
    }

    @Bean
    public ChatModelObservationConvention chatModelObservationConvention() {
        return new DefaultChatModelObservationConvention();
    }
}

Creating Chat Observations

import io.micrometer.observation.Observation;
import org.springframework.ai.chat.observation.ChatModelObservationContext;

@Service
public class ObservedChatService {
    private final ChatModel chatModel;
    private final ObservationRegistry observationRegistry;

    public String chat(String message) {
        // Create prompt
        Prompt prompt = new Prompt(message);

        // Create observation context
        ChatModelObservationContext context =
            new ChatModelObservationContext(prompt, chatModel);

        // Observe the operation
        return Observation.createNotStarted("chat.model.call", context, observationRegistry)
            .observe(() -> {
                ChatResponse response = chatModel.call(prompt);
                context.setResponse(response);
                return response.getResult().getOutput().getText();
            });
    }
}

Embedding Model Observability

import org.springframework.ai.embedding.observation.*;

@Service
public class ObservedEmbeddingService {
    private final EmbeddingModel embeddingModel;
    private final ObservationRegistry observationRegistry;

    public float[] embedWithObservation(String text) {
        EmbeddingRequest request = new EmbeddingRequest(
            List.of(text),
            null
        );

        EmbeddingModelObservationContext context =
            new EmbeddingModelObservationContext(request, embeddingModel);

        return Observation.createNotStarted("embedding.model.call", context, observationRegistry)
            .observe(() -> {
                EmbeddingResponse response = embeddingModel.call(request);
                context.setResponse(response);
                return response.getResult().getOutput();
            });
    }
}

Tool Calling Observability

import org.springframework.ai.tool.observation.*;

public class ObservedToolCallback implements ToolCallback {
    private final ToolCallback delegate;
    private final ObservationRegistry observationRegistry;

    @Override
    public String call(String toolInput, ToolContext toolContext) {
        ToolCallingObservationContext context =
            new ToolCallingObservationContext(getToolDefinition(), toolInput);

        return Observation.createNotStarted("tool.call", context, observationRegistry)
            .observe(() -> {
                try {
                    String result = delegate.call(toolInput, toolContext);
                    context.setToolResult(result);
                    return result;
                } catch (Exception e) {
                    context.setError(e);
                    throw e;
                }
            });
    }
}

Custom Observation Handler

import io.micrometer.observation.Observation;
import io.micrometer.observation.ObservationHandler;

public class CustomChatObservationHandler
    implements ObservationHandler<ChatModelObservationContext> {

    @Override
    public void onStart(ChatModelObservationContext context) {
        System.out.println("Chat started: " + context.getRequest().getContents());
    }

    @Override
    public void onStop(ChatModelObservationContext context) {
        ChatResponse response = context.getResponse();
        if (response != null) {
            System.out.println("Tokens used: " +
                response.getMetadata().getUsage().getTotalTokens());
        }
    }

    @Override
    public void onError(ChatModelObservationContext context) {
        System.err.println("Chat failed: " + context.getError());
    }

    @Override
    public boolean supportsContext(Observation.Context context) {
        return context instanceof ChatModelObservationContext;
    }
}

Metrics Collection

import io.micrometer.core.instrument.MeterRegistry;
import io.micrometer.core.instrument.Timer;

@Service
public class MetricsCollectingChatService {
    private final ChatModel chatModel;
    private final MeterRegistry meterRegistry;

    public String chat(String message) {
        Timer.Sample sample = Timer.start(meterRegistry);

        try {
            ChatResponse response = chatModel.call(new Prompt(message));

            // Record success metrics
            sample.stop(Timer.builder("chat.model.duration")
                .tag("status", "success")
                .tag("model", response.getMetadata().getModel())
                .register(meterRegistry));

            // Record token usage
            Usage usage = response.getMetadata().getUsage();
            meterRegistry.counter("chat.model.tokens",
                "type", "prompt").increment(usage.getPromptTokens());
            meterRegistry.counter("chat.model.tokens",
                "type", "completion").increment(usage.getCompletionTokens());

            return response.getResult().getOutput().getText();
        } catch (Exception e) {
            sample.stop(Timer.builder("chat.model.duration")
                .tag("status", "error")
                .register(meterRegistry));
            throw e;
        }
    }
}

Distributed Tracing

import io.micrometer.tracing.Tracer;
import io.micrometer.tracing.Span;

@Service
public class TracedChatService {
    private final ChatModel chatModel;
    private final Tracer tracer;

    public String chat(String message) {
        Span span = tracer.nextSpan().name("chat.model.call");

        try (Tracer.SpanInScope ws = tracer.withSpan(span.start())) {
            span.tag("message.length", String.valueOf(message.length()));

            ChatResponse response = chatModel.call(new Prompt(message));

            span.tag("tokens.total",
                String.valueOf(response.getMetadata().getUsage().getTotalTokens()));
            span.tag("model",
                response.getMetadata().getModel());

            return response.getResult().getOutput().getText();
        } finally {
            span.end();
        }
    }
}

Logging Handler

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class LoggingChatObservationHandler
    implements ObservationHandler<ChatModelObservationContext> {

    private static final Logger log = LoggerFactory.getLogger(
        LoggingChatObservationHandler.class
    );

    @Override
    public void onStart(ChatModelObservationContext context) {
        log.info("Starting chat model call");
    }

    @Override
    public void onStop(ChatModelObservationContext context) {
        ChatResponse response = context.getResponse();
        if (response != null) {
            ChatResponseMetadata metadata = response.getMetadata();
            log.info("Chat completed - Model: {}, Tokens: {}",
                metadata.getModel(),
                metadata.getUsage().getTotalTokens()
            );
        }
    }

    @Override
    public void onError(ChatModelObservationContext context) {
        log.error("Chat failed", context.getError());
    }

    @Override
    public boolean supportsContext(Observation.Context context) {
        return context instanceof ChatModelObservationContext;
    }
}

Spring Boot Integration

@Configuration
public class ObservabilityAutoConfiguration {

    @Bean
    @ConditionalOnMissingBean
    public ChatModelObservationConvention chatModelObservationConvention() {
        return new DefaultChatModelObservationConvention();
    }

    @Bean
    public ChatModelCompletionObservationHandler chatModelCompletionHandler() {
        return new ChatModelCompletionObservationHandler();
    }

    @Bean
    @ConditionalOnProperty(name = "spring.ai.chat.observations.include-prompt", havingValue = "true")
    public ChatModelPromptContentObservationHandler promptContentHandler() {
        return new ChatModelPromptContentObservationHandler();
    }

    @Bean
    public ChatModelMeterObservationHandler chatModelMeterHandler(
        MeterRegistry meterRegistry
    ) {
        return new ChatModelMeterObservationHandler(meterRegistry);
    }
}

Complete Observability Setup

@Configuration
public class CompleteObservabilityConfig {

    @Bean
    public ObservationRegistry observationRegistry(
        MeterRegistry meterRegistry,
        Tracer tracer
    ) {
        ObservationRegistry registry = ObservationRegistry.create();

        // Configure handlers
        registry.observationConfig()
            // Chat model handlers
            .observationHandler(new ChatModelCompletionObservationHandler())
            .observationHandler(new ChatModelMeterObservationHandler(meterRegistry))
            .observationHandler(new ChatModelPromptContentObservationHandler())
            // Embedding handlers
            .observationHandler(new EmbeddingModelMeterObservationHandler(meterRegistry))
            // Tool handlers
            .observationHandler(new ToolCallingContentObservationFilter())
            // Error handler
            .observationHandler(new ErrorLoggingObservationHandler())
            // Tracing
            .observationHandler(new DefaultTracingObservationHandler(tracer));

        return registry;
    }

    @Bean
    public ObservationRegistryCustomizer observationCustomizer() {
        return registry -> {
            registry.observationConfig()
                .observationConvention(new DefaultChatModelObservationConvention())
                .observationConvention(new DefaultEmbeddingModelObservationConvention())
                .observationConvention(new DefaultToolCallingObservationConvention());
        };
    }
}

Install with Tessl CLI

npx tessl i tessl/maven-org-springframework-ai--spring-ai-model

tessl/maven-org-springframework-ai--spring-ai-model

observability.md.css-3qkkll{font-size:var(--chakra-font-sizes-sm);font-weight:var(--chakra-font-weights-normal);color:var(--chakra-colors-gray-300);}docs/reference/

Observability

Core Observability Types

ModelObservationContext

ErrorLoggingObservationHandler

ModelUsageMetricsGenerator

Capabilities

Chat Model Observability

ChatModelObservationContext

ChatModelObservationConvention Interface

DefaultChatModelObservationConvention

ChatModelObservationHandler Implementations

ChatModelObservationDocumentation

Embedding Model Observability

EmbeddingModelObservationContext

EmbeddingModelObservationConvention Interface

DefaultEmbeddingModelObservationConvention

EmbeddingModelMeterObservationHandler

EmbeddingModelObservationDocumentation

Image Model Observability

ImageModelObservationContext

ImageModelObservationConvention Interface

DefaultImageModelObservationConvention

ImageModelPromptContentObservationHandler

ImageModelObservationDocumentation

Observation Key Names

Chat Model Observation Keys

Embedding Model Observation Keys

Image Model Observation Keys

Tool Calling Observation Keys

Tool Calling Observability

ToolCallingObservationContext

ToolCallingObservationConvention Interface

DefaultToolCallingObservationConvention

ToolCallingContentObservationFilter

ToolCallingObservationDocumentation

Model Observability Base

ModelObservationContext

ErrorLoggingObservationHandler

ModelUsageMetricsGenerator

Usage Examples

Enabling Chat Model Observability

Creating Chat Observations

Embedding Model Observability

Tool Calling Observability

Custom Observation Handler

Metrics Collection

Distributed Tracing

Logging Handler

Spring Boot Integration

Complete Observability Setup

observability.mddocs/reference/