CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

tessl/maven-org-springframework-ai--spring-ai-ollama

Spring Boot-compatible Ollama integration providing ChatModel and EmbeddingModel implementations for running large language models locally with support for streaming, tool calling, model management, and observability.

Overview
Eval results
Files

tool-calling.mddocs/reference/

Tool Calling and Function Support

Enable models to call external functions and tools.

Overview

Ollama supports tool/function calling with compatible models (Llama 3, Mistral, Qwen, etc.). Spring AI provides automatic tool execution, JSON Schema generation, and seamless integration with your Java methods.

Supported Models

Tool calling requires compatible models:

  • Llama 3 family (llama3, llama3.1, llama3.2)
  • Mistral family (mistral, mistral-nemo)
  • Qwen family (qwen2.5, qwen3)

Check model documentation for tool support before use.

Basic Tool Calling

Defining a Tool

Create a service class with tool methods:

public class WeatherService {

    public record Request(String location, String unit) {}

    public record Response(String location, String temperature, String unit) {}

    public Response getWeather(Request request) {
        // Simulated weather lookup
        String temp = switch (request.location().toLowerCase()) {
            case "san francisco" -> "65";
            case "tokyo" -> "50";
            case "paris" -> "59";
            default -> "72";
        };

        return new Response(
            request.location(),
            temp,
            request.unit()
        );
    }
}

Registering Tools

Register tools using FunctionToolCallback:

WeatherService weatherService = new WeatherService();

OllamaChatOptions options = OllamaChatOptions.builder()
    .model(OllamaModel.LLAMA3)
    .toolCallbacks(List.of(
        FunctionToolCallback.builder("getWeather", weatherService)
            .description("Get current weather for a location")
            .inputType(WeatherService.Request.class)
            .build()
    ))
    .build();

OllamaChatModel chatModel = OllamaChatModel.builder()
    .ollamaApi(ollamaApi)
    .defaultOptions(options)
    .build();

Using Tools

String prompt = "What's the weather in San Francisco and Tokyo? Use Celsius.";

ChatResponse response = chatModel.call(new Prompt(prompt));

// Model automatically calls getWeather tool and includes results in response
System.out.println(response.getResult().getOutput().getText());
// Output: "The weather in San Francisco is 65°C and in Tokyo is 50°C"

Tool Configuration

Tool Callbacks

Tools are registered via ToolCallback (usually FunctionToolCallback).

// Basic tool registration
FunctionToolCallback weatherTool = FunctionToolCallback.builder(
    "getWeather",
    weatherService
)
    .description("Get weather information for a location")
    .inputType(WeatherService.Request.class)
    .build();

OllamaChatOptions options = OllamaChatOptions.builder()
    .toolCallbacks(List.of(weatherTool))
    .build();

Multiple Tools

Register multiple tools for the model to choose from:

OllamaChatOptions options = OllamaChatOptions.builder()
    .model(OllamaModel.LLAMA3)
    .toolCallbacks(List.of(
        FunctionToolCallback.builder("getWeather", weatherService)
            .description("Get current weather")
            .inputType(WeatherRequest.class)
            .build(),
        FunctionToolCallback.builder("getTime", timeService)
            .description("Get current time in a timezone")
            .inputType(TimeRequest.class)
            .build(),
        FunctionToolCallback.builder("searchWeb", searchService)
            .description("Search the web for information")
            .inputType(SearchRequest.class)
            .build()
    ))
    .build();

// Model will choose appropriate tool(s) based on the prompt
String prompt = "What's the weather in London and what time is it there?";
ChatResponse response = chatModel.call(new Prompt(prompt));

Selective Tool Enablement

Enable only specific tools for a request:

// Register all tools at model level
OllamaChatModel chatModel = OllamaChatModel.builder()
    .ollamaApi(ollamaApi)
    .defaultOptions(OllamaChatOptions.builder()
        .model(OllamaModel.LLAMA3)
        .toolCallbacks(List.of(weatherTool, timeTool, searchTool))
        .build())
    .build();

// Enable only weather tool for this request
OllamaChatOptions requestOptions = OllamaChatOptions.builder()
    .toolNames("getWeather")  // Only enable weather tool
    .build();

ChatResponse response = chatModel.call(
    new Prompt("What's the weather?", requestOptions)
);

Tool Context

Share data between tool executions:

OllamaChatOptions options = OllamaChatOptions.builder()
    .model(OllamaModel.LLAMA3)
    .toolCallbacks(List.of(weatherTool))
    .toolContext(Map.of(
        "apiKey", "xyz123",
        "userId", "user-456",
        "maxResults", 10
    ))
    .build();

// Tools can access context during execution
public class WeatherService {
    public Response getWeather(Request request, Map<String, Object> context) {
        String apiKey = (String) context.get("apiKey");
        // Use apiKey for API calls
        return fetchWeather(request.location(), apiKey);
    }
}

Internal Tool Execution

Control whether tools are executed automatically:

// Default: Auto-execute tools (recommended)
OllamaChatOptions options = OllamaChatOptions.builder()
    .toolCallbacks(List.of(weatherTool))
    .internalToolExecutionEnabled(true)  // Default behavior
    .build();

// Disable auto-execution (manual handling)
OllamaChatOptions options = OllamaChatOptions.builder()
    .toolCallbacks(List.of(weatherTool))
    .internalToolExecutionEnabled(false)
    .build();

// Tool calls will be in response but not executed
ChatResponse response = chatModel.call(new Prompt("Get weather"));
// Check for tool calls in response and handle manually

Advanced Usage

Streaming with Tools

Tool calls work with streaming responses:

OllamaChatOptions options = OllamaChatOptions.builder()
    .model(OllamaModel.LLAMA3)
    .toolCallbacks(List.of(weatherTool))
    .build();

Flux<ChatResponse> stream = chatModel.stream(new Prompt(
    "What's the weather in San Francisco?",
    options
));

stream.subscribe(chunk -> {
    String content = chunk.getResult().getOutput().getText();
    if (content != null) {
        System.out.print(content);
    }
});
// Model will call tool during streaming and incorporate results

Complex Tool Definitions

Define tools with complex parameter schemas:

public class SearchService {

    public record Request(
        String query,
        List<String> domains,
        int maxResults,
        boolean includeImages,
        Map<String, String> filters
    ) {}

    public record Result(
        String title,
        String url,
        String snippet,
        List<String> images
    ) {}

    public List<Result> search(Request request) {
        // Complex search logic
        return performSearch(request);
    }
}

FunctionToolCallback searchTool = FunctionToolCallback.builder(
    "searchWeb",
    searchService
)
    .description("Search the web with advanced filters")
    .inputType(SearchService.Request.class)  // Auto-generates schema
    .build();

Prompt-Level Tool Override

Override tools for specific prompts:

// Model has default tools
OllamaChatModel chatModel = OllamaChatModel.builder()
    .ollamaApi(ollamaApi)
    .defaultOptions(OllamaChatOptions.builder()
        .model(OllamaModel.LLAMA3)
        .toolCallbacks(List.of(weatherTool, timeTool))
        .build())
    .build();

// Add additional tool for this request only
OllamaChatOptions requestOptions = OllamaChatOptions.builder()
    .toolCallbacks(List.of(specialTool))
    .build();

// This request has access to weather, time, AND special tool
ChatResponse response = chatModel.call(
    new Prompt("Use the special tool", requestOptions)
);

Tool Execution Eligibility

Control which tools can be executed based on custom logic:

ToolExecutionEligibilityPredicate predicate = (tool, context) -> {
    // Only allow weather tool during business hours
    LocalTime now = LocalTime.now();
    if (tool.getName().equals("getWeather")) {
        return now.getHour() >= 9 && now.getHour() <= 17;
    }
    return true;
};

OllamaChatModel chatModel = OllamaChatModel.builder()
    .ollamaApi(ollamaApi)
    .defaultOptions(OllamaChatOptions.builder()
        .model(OllamaModel.LLAMA3)
        .toolCallbacks(List.of(weatherTool))
        .build())
    .toolExecutionEligibilityPredicate(predicate)
    .build();

Tool Call Inspection

Examine tool calls in the response:

ChatResponse response = chatModel.call(new Prompt("Get weather"));

AssistantMessage message = response.getResult().getOutput();

// Check if message includes tool calls
List<ToolCall> toolCalls = message.getToolCalls();

if (!toolCalls.isEmpty()) {
    for (ToolCall toolCall : toolCalls) {
        String functionName = toolCall.name();
        Map<String, Object> arguments = toolCall.arguments();
        System.out.println("Called: " + functionName + " with " + arguments);
    }
}

Manual Tool Execution

Disable automatic execution and handle tools manually:

OllamaChatOptions options = OllamaChatOptions.builder()
    .model(OllamaModel.LLAMA3)
    .toolCallbacks(List.of(weatherTool))
    .internalToolExecutionEnabled(false)  // Disable auto-execution
    .build();

ChatResponse response = chatModel.call(new Prompt("Get weather", options));

// Extract tool calls
AssistantMessage message = response.getResult().getOutput();
List<ToolCall> toolCalls = message.getToolCalls();

// Execute manually
for (ToolCall toolCall : toolCalls) {
    String result = executeToolManually(toolCall);

    // Create tool response message
    ToolResponseMessage toolResponse = new ToolResponseMessage(
        result,
        toolCall.name()
    );

    // Continue conversation with tool results
    ChatResponse nextResponse = chatModel.call(new Prompt(List.of(
        message,
        toolResponse
    )));
}

Tool Definition Schema

Tools are defined using JSON Schema. Spring AI auto-generates schemas from Java classes:

// Input class
public record WeatherRequest(
    String location,
    String unit
) {}

// Becomes JSON Schema:
{
    "type": "object",
    "properties": {
        "location": {
            "type": "string",
            "description": "Location to get weather for"
        },
        "unit": {
            "type": "string",
            "description": "Temperature unit (celsius or fahrenheit)"
        }
    },
    "required": ["location", "unit"]
}

Custom JSON Schema

Provide custom schemas if needed:

String customSchema = """
{
    "type": "object",
    "properties": {
        "location": {
            "type": "string",
            "description": "City name"
        },
        "unit": {
            "type": "string",
            "enum": ["celsius", "fahrenheit"]
        }
    },
    "required": ["location"]
}
""";

FunctionToolCallback tool = FunctionToolCallback.builder(
    "getWeather",
    weatherService
)
    .description("Get weather")
    .schemaType(SchemaType.OPEN_API_SCHEMA)
    .inputType(customSchema)  // Use custom schema
    .build();

Complete Examples

Weather Service Example

// Service implementation
public class WeatherService {

    public record Request(String location, String unit) {}

    public record Response(String location, double temperature, String unit, String conditions) {
        @Override
        public String toString() {
            return String.format("%s: %.1f°%s, %s",
                location, temperature, unit.substring(0, 1).toUpperCase(), conditions);
        }
    }

    public Response getWeather(Request request) {
        // Simulate weather data
        return new Response(
            request.location(),
            getTemperature(request.location(), request.unit()),
            request.unit(),
            getConditions(request.location())
        );
    }

    private double getTemperature(String location, String unit) {
        double celsius = switch (location.toLowerCase()) {
            case "san francisco" -> 18.3;
            case "tokyo" -> 10.5;
            case "paris" -> 15.2;
            default -> 22.0;
        };

        return unit.equalsIgnoreCase("fahrenheit")
            ? (celsius * 9/5) + 32
            : celsius;
    }

    private String getConditions(String location) {
        return switch (location.toLowerCase()) {
            case "san francisco" -> "Foggy";
            case "tokyo" -> "Clear";
            case "paris" -> "Partly Cloudy";
            default -> "Sunny";
        };
    }
}

// Configuration
@Configuration
public class ToolConfig {

    @Bean
    public WeatherService weatherService() {
        return new WeatherService();
    }

    @Bean
    public OllamaChatModel chatModel(OllamaApi ollamaApi, WeatherService weatherService) {
        return OllamaChatModel.builder()
            .ollamaApi(ollamaApi)
            .defaultOptions(OllamaChatOptions.builder()
                .model(OllamaModel.LLAMA3)
                .temperature(0.7)
                .toolCallbacks(List.of(
                    FunctionToolCallback.builder("getWeather", weatherService)
                        .description("Get current weather for a location")
                        .inputType(WeatherService.Request.class)
                        .build()
                ))
                .build())
            .build();
    }
}

// Usage
@Service
public class ChatService {

    private final OllamaChatModel chatModel;

    public String chat(String userMessage) {
        ChatResponse response = chatModel.call(new Prompt(userMessage));
        return response.getResult().getOutput().getText();
    }
}

// Example queries:
// "What's the weather in San Francisco?"
// "Compare temperatures in Tokyo and Paris"
// "Is it warmer in San Francisco or Tokyo? Use Celsius"

Multi-Tool Example

// Multiple services
public class WeatherService {
    public record Request(String location, String unit) {}
    public record Response(String location, double temp, String unit) {}

    public Response getWeather(Request request) {
        // Implementation
    }
}

public class TimeService {
    public record Request(String timezone) {}
    public record Response(String timezone, String time, String date) {}

    public Response getTime(Request request) {
        // Implementation
    }
}

public class CurrencyService {
    public record Request(String from, String to, double amount) {}
    public record Response(String from, String to, double result) {}

    public Response convertCurrency(Request request) {
        // Implementation
    }
}

// Configuration with multiple tools
OllamaChatModel chatModel = OllamaChatModel.builder()
    .ollamaApi(ollamaApi)
    .defaultOptions(OllamaChatOptions.builder()
        .model(OllamaModel.LLAMA3)
        .toolCallbacks(List.of(
            FunctionToolCallback.builder("getWeather", weatherService)
                .description("Get weather information")
                .inputType(WeatherService.Request.class)
                .build(),
            FunctionToolCallback.builder("getTime", timeService)
                .description("Get current time in a timezone")
                .inputType(TimeService.Request.class)
                .build(),
            FunctionToolCallback.builder("convertCurrency", currencyService)
                .description("Convert between currencies")
                .inputType(CurrencyService.Request.class)
                .build()
        ))
        .build())
    .build();

// Model chooses appropriate tools:
// "What's the weather in London and what time is it there?"
// "How much is 100 USD in EUR?"
// "Give me weather for Paris and convert 50 EUR to USD"

Best Practices

  1. Clear Descriptions: Provide clear tool and parameter descriptions for better model understanding
  2. Type Safety: Use strongly-typed request/response records
  3. Error Handling: Handle tool execution errors gracefully
  4. Tool Context: Use tool context for shared state (API keys, config)
  5. Selective Enablement: Only enable tools needed for each request
  6. Test Tool Compatibility: Verify your model supports tool calling
  7. Monitor Tool Usage: Log tool calls for debugging
  8. Validate Inputs: Tools should validate parameters before execution
  9. Return Structured Data: Return structured responses for better model understanding
  10. Keep Tools Focused: Each tool should do one thing well

Limitations

  1. Model Support: Not all Ollama models support tool calling
  2. Tool Call Accuracy: Models may not always choose the right tool or provide correct parameters
  3. Error Recovery: Limited model ability to recover from tool execution errors
  4. Nested Tools: Tools cannot directly call other tools
  5. Async Tools: Tool callbacks must be synchronous
  6. Streaming Complexity: Tool calls in streaming responses require special handling

Related Documentation

  • OllamaChatOptions - Tool configuration options
  • OllamaChatModel - Using the chat model
  • OllamaModel - Models supporting tool calling
  • API Types - Tool-related types (ToolCall, ToolCallFunction, etc.)

Notes

  1. Tool callbacks are registered at the options level
  2. Tools can be registered globally (default options) or per-request (prompt options)
  3. Tool context is shared across all tool executions in a conversation
  4. Internal tool execution is enabled by default
  5. Tool eligibility predicates apply before execution
  6. Tools receive deserialized JSON arguments matching their input type
  7. Tool responses are automatically converted to messages and added to conversation history
tessl i tessl/maven-org-springframework-ai--spring-ai-ollama@1.1.1

docs

reference

api-client.md

api-types.md

chat-model.md

chat-options.md

embedding-model.md

embedding-options.md

error-handling.md

model-management.md

models.md

multimodal.md

observability.md

thinking.md

tool-calling.md

index.md

tile.json