CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

tessl/maven-org-springframework-ai--spring-ai-ollama

Spring Boot-compatible Ollama integration providing ChatModel and EmbeddingModel implementations for running large language models locally with support for streaming, tool calling, model management, and observability.

Overview
Eval results
Files

model-management.mddocs/reference/

Model Management

Automatic model lifecycle management for Ollama.

Overview

Spring AI Ollama provides automatic model management through OllamaModelManager, ModelManagementOptions, and PullModelStrategy. These components handle pulling, checking availability, and deleting models.

OllamaModelManager

Manages the lifecycle of Ollama models.

Class Information

package org.springframework.ai.ollama.management;

public class OllamaModelManager

Creating a Manager

// With default options (NEVER pull strategy)
OllamaModelManager manager = new OllamaModelManager(ollamaApi);

// With custom options
ModelManagementOptions options = ModelManagementOptions.builder()
    .pullModelStrategy(PullModelStrategy.WHEN_MISSING)
    .timeout(Duration.ofMinutes(10))
    .build();

OllamaModelManager manager = new OllamaModelManager(ollamaApi, options);

Constructors:

public OllamaModelManager(OllamaApi ollamaApi)
public OllamaModelManager(OllamaApi ollamaApi, ModelManagementOptions options)

Checking Model Availability

Check if a model is available locally.

boolean available = manager.isModelAvailable("llama3");

if (!available) {
    System.out.println("Model not found locally");
}

// Works with version tags
boolean availableWithTag = manager.isModelAvailable("llama3:latest");

Method:

public boolean isModelAvailable(String modelName)

Returns: true if the model exists locally, false otherwise

Note: Model names are normalized - "llama3" becomes "llama3:latest" internally.

Pulling Models

Pull a model from the Ollama library.

// Pull using configured strategy
manager.pullModel("llama3");

// Pull with specific strategy
manager.pullModel("llama3", PullModelStrategy.ALWAYS);

Methods:

public void pullModel(String modelName)
public void pullModel(String modelName, PullModelStrategy strategy)

Behavior:

  • NEVER: Does nothing
  • WHEN_MISSING: Pulls only if not already available
  • ALWAYS: Pulls the latest version regardless

Progress Logging:

The manager logs pull progress:

INFO: Start pulling model: llama3
INFO: Pulling the 'llama3' model - Status: downloading
INFO: Pulling the 'llama3' model - Status: success
INFO: Completed pulling the 'llama3' model

Deleting Models

Delete a model from local storage.

manager.deleteModel("old-model");

Method:

public void deleteModel(String modelName)

Behavior:

  • Checks if model exists before attempting deletion
  • Logs the deletion process
  • No-op if model doesn't exist

ModelManagementOptions

Configuration for model management behavior.

Class Information

package org.springframework.ai.ollama.management;

public record ModelManagementOptions(
    PullModelStrategy pullModelStrategy,
    List<String> additionalModels,
    Duration timeout,
    Integer maxRetries
)

Creating Options

// Use defaults
ModelManagementOptions options = ModelManagementOptions.defaults();

// Using builder
ModelManagementOptions options = ModelManagementOptions.builder()
    .pullModelStrategy(PullModelStrategy.WHEN_MISSING)
    .additionalModels(List.of("llama3", "mistral"))
    .timeout(Duration.ofMinutes(10))
    .maxRetries(3)
    .build();

Configuration Fields

pullModelStrategy

Strategy for pulling models.

.pullModelStrategy(PullModelStrategy.WHEN_MISSING)

Type: PullModelStrategy Default: PullModelStrategy.NEVER

See PullModelStrategy section below.

additionalModels

List of models to pull automatically during initialization.

.additionalModels(List.of("llama3", "mistral", "nomic-embed-text"))

Type: List<String> Default: Empty list

When OllamaModelManager is created, it automatically pulls all models in this list using the configured strategy.

timeout

Timeout for model pull operations.

.timeout(Duration.ofMinutes(15))

Type: Duration Default: Duration.ofMinutes(5)

Large models may require longer timeouts.

maxRetries

Maximum number of retry attempts for failed pulls.

.maxRetries(3)

Type: Integer Default: 0 (no retries)

Uses exponential backoff starting at 5 seconds.

Default Options

ModelManagementOptions defaults = ModelManagementOptions.defaults();
// pullModelStrategy: NEVER
// additionalModels: []
// timeout: 5 minutes
// maxRetries: 0

PullModelStrategy

Strategy enum for controlling when models are pulled.

Class Information

package org.springframework.ai.ollama.management;

public enum PullModelStrategy {
    ALWAYS,
    WHEN_MISSING,
    NEVER
}

Strategies

ALWAYS

Always pull the model, even if it's already available.

.pullModelStrategy(PullModelStrategy.ALWAYS)

Use case: Ensure you're using the latest version of a model that may have been updated.

Example:

ModelManagementOptions options = ModelManagementOptions.builder()
    .pullModelStrategy(PullModelStrategy.ALWAYS)
    .additionalModels(List.of("llama3"))
    .build();

// Will pull llama3 every time, even if it exists
OllamaModelManager manager = new OllamaModelManager(ollamaApi, options);

WHEN_MISSING

Only pull the model if it's not already available locally.

.pullModelStrategy(PullModelStrategy.WHEN_MISSING)

Use case: Default strategy for most applications - ensures model is available without unnecessary downloads.

Example:

ModelManagementOptions options = ModelManagementOptions.builder()
    .pullModelStrategy(PullModelStrategy.WHEN_MISSING)
    .additionalModels(List.of("llama3", "mistral"))
    .build();

// Only pulls models that don't exist locally
OllamaModelManager manager = new OllamaModelManager(ollamaApi, options);

NEVER

Never pull models automatically.

.pullModelStrategy(PullModelStrategy.NEVER)

Use case: Production environments where models are pre-installed, or you want explicit control.

Example:

ModelManagementOptions options = ModelManagementOptions.builder()
    .pullModelStrategy(PullModelStrategy.NEVER)
    .build();

// Manager won't pull any models automatically
OllamaModelManager manager = new OllamaModelManager(ollamaApi, options);

// But you can still pull manually with a different strategy
manager.pullModel("llama3", PullModelStrategy.WHEN_MISSING);

Usage Examples

Automatic Model Initialization

Ensure required models are available on startup.

ModelManagementOptions options = ModelManagementOptions.builder()
    .pullModelStrategy(PullModelStrategy.WHEN_MISSING)
    .additionalModels(List.of(
        "llama3",
        "mistral",
        "nomic-embed-text"
    ))
    .timeout(Duration.ofMinutes(15))
    .maxRetries(2)
    .build();

OllamaModelManager manager = new OllamaModelManager(ollamaApi, options);
// Models are pulled during construction if missing

Integration with ChatModel

OllamaApi ollamaApi = OllamaApi.builder()
    .baseUrl("http://localhost:11434")
    .build();

ModelManagementOptions managementOptions = ModelManagementOptions.builder()
    .pullModelStrategy(PullModelStrategy.WHEN_MISSING)
    .build();

OllamaChatModel chatModel = OllamaChatModel.builder()
    .ollamaApi(ollamaApi)
    .defaultOptions(OllamaChatOptions.builder()
        .model("llama3")
        .build())
    .modelManagementOptions(managementOptions)
    .build();

// Model is automatically pulled if missing before first use

Dynamic Model Management

Check and pull models on demand.

OllamaModelManager manager = new OllamaModelManager(ollamaApi);

String modelName = "llama3";

if (!manager.isModelAvailable(modelName)) {
    System.out.println("Model not found, pulling...");
    manager.pullModel(modelName, PullModelStrategy.ALWAYS);
    System.out.println("Model ready");
}

// Use the model
OllamaChatModel chatModel = OllamaChatModel.builder()
    .ollamaApi(ollamaApi)
    .defaultOptions(OllamaChatOptions.builder().model(modelName).build())
    .build();

Model Version Updates

Force update to the latest model version.

OllamaModelManager manager = new OllamaModelManager(ollamaApi);

// Pull latest version even if model exists
manager.pullModel("llama3", PullModelStrategy.ALWAYS);

// Now using the latest version

Cleanup Old Models

Remove models you no longer need.

OllamaModelManager manager = new OllamaModelManager(ollamaApi);

// List all models
ListModelResponse response = ollamaApi.listModels();

for (Model model : response.models()) {
    if (model.name().startsWith("old-")) {
        System.out.println("Deleting " + model.name());
        manager.deleteModel(model.name());
    }
}

Production Configuration

Conservative settings for production.

ModelManagementOptions options = ModelManagementOptions.builder()
    // Don't auto-pull in production
    .pullModelStrategy(PullModelStrategy.NEVER)
    // Long timeout for manual pulls if needed
    .timeout(Duration.ofMinutes(30))
    // Retry network failures
    .maxRetries(5)
    .build();

OllamaModelManager manager = new OllamaModelManager(ollamaApi, options);

// Manually ensure required models exist
List<String> requiredModels = List.of("llama3", "mistral");
for (String model : requiredModels) {
    if (!manager.isModelAvailable(model)) {
        throw new RuntimeException("Required model not available: " + model);
    }
}

Development Configuration

Convenient settings for development.

ModelManagementOptions options = ModelManagementOptions.builder()
    // Auto-pull missing models
    .pullModelStrategy(PullModelStrategy.WHEN_MISSING)
    // Preload common models
    .additionalModels(List.of(
        "llama3",
        "mistral",
        "nomic-embed-text",
        "mxbai-embed-large"
    ))
    // Shorter timeout for faster feedback
    .timeout(Duration.ofMinutes(10))
    .maxRetries(1)
    .build();

OllamaModelManager manager = new OllamaModelManager(ollamaApi, options);
// All models ready to use

Spring Boot Configuration

Using Spring Boot auto-configuration.

@Configuration
public class OllamaConfig {

    @Bean
    public OllamaApi ollamaApi() {
        return OllamaApi.builder()
            .baseUrl("http://localhost:11434")
            .build();
    }

    @Bean
    public ModelManagementOptions modelManagementOptions() {
        return ModelManagementOptions.builder()
            .pullModelStrategy(PullModelStrategy.WHEN_MISSING)
            .additionalModels(List.of("llama3", "mistral"))
            .timeout(Duration.ofMinutes(15))
            .maxRetries(2)
            .build();
    }

    @Bean
    public OllamaModelManager ollamaModelManager(
            OllamaApi ollamaApi,
            ModelManagementOptions options) {
        return new OllamaModelManager(ollamaApi, options);
    }

    @Bean
    public OllamaChatModel chatModel(
            OllamaApi ollamaApi,
            ModelManagementOptions options) {
        return OllamaChatModel.builder()
            .ollamaApi(ollamaApi)
            .defaultOptions(OllamaChatOptions.builder()
                .model("llama3")
                .build())
            .modelManagementOptions(options)
            .build();
    }
}

Model Name Normalization

Model names are automatically normalized:

// These are equivalent:
manager.isModelAvailable("llama3");
manager.isModelAvailable("llama3:latest");

// Version-specific checks:
manager.isModelAvailable("llama3:70b");
manager.isModelAvailable("mistral:7b");

Normalization rules:

  • If name contains :, use as-is
  • If name doesn't contain :, append :latest

Progress and Logging

The manager logs operations at INFO level:

INFO: Start pulling model: llama3
INFO: Pulling the 'llama3' model - Status: pulling manifest
INFO: Pulling the 'llama3' model - Status: downloading
INFO: Pulling the 'llama3' model - Status: verifying sha256 digest
INFO: Pulling the 'llama3' model - Status: success
INFO: Completed pulling the 'llama3' model

INFO: Start deletion of model: old-model
INFO: Completed deletion of model: old-model

DEBUG level provides additional details:

DEBUG: Model 'llama3' already available. Skipping pull operation.

Error Handling

Timeout Errors

ModelManagementOptions options = ModelManagementOptions.builder()
    .timeout(Duration.ofMinutes(5))
    .build();

OllamaModelManager manager = new OllamaModelManager(ollamaApi, options);

try {
    manager.pullModel("very-large-model");
} catch (TimeoutException e) {
    // Handle timeout - model too large for timeout
}

Network Errors with Retry

ModelManagementOptions options = ModelManagementOptions.builder()
    .maxRetries(3)  // Retry up to 3 times
    .build();

// Automatic exponential backoff retry on network failures
manager.pullModel("llama3");

Model Not Found

if (!manager.isModelAvailable("unknown-model")) {
    System.err.println("Model not found");
    // Pull or handle accordingly
}

Best Practices

  1. Use WHEN_MISSING in development - Convenient and safe
  2. Use NEVER in production - Explicit control, faster startup
  3. Set appropriate timeouts - Large models need more time
  4. Enable retries for network reliability - Handles transient failures
  5. Check availability before use - Fail fast if models missing
  6. Use additionalModels for required dependencies - Ensures all needed models are present
  7. Monitor model sizes - Use ListModelResponse to track disk usage

Related Documentation

  • OllamaChatModel - Chat model with automatic management
  • OllamaEmbeddingModel - Embedding model with automatic management
  • OllamaApi - Low-level model management methods
  • API Types - Model management request/response types

Notes

  1. Model pulls use streaming for progress updates
  2. Pulls are blocked/synchronous operations (use blockLast() internally)
  3. Model names are case-sensitive
  4. The :latest tag is implicit when not specified
  5. Pulling large models (70B+) can take significant time and disk space
  6. Model manager is thread-safe
  7. additionalModels are pulled during manager construction, blocking initialization
tessl i tessl/maven-org-springframework-ai--spring-ai-ollama@1.1.1

docs

reference

api-client.md

api-types.md

chat-model.md

chat-options.md

embedding-model.md

embedding-options.md

error-handling.md

model-management.md

models.md

multimodal.md

observability.md

thinking.md

tool-calling.md

index.md

tile.json