Quarkus LangChain4j Ollama Deployment

The Quarkus LangChain4j Ollama Deployment module is a build-time extension that enables seamless integration of Ollama LLM models with Quarkus applications through the LangChain4j framework. This deployment module performs build-time processing to configure Ollama-based chat models, embedding models, and streaming chat models, with automatic DevServices provisioning for development environments.

Package Information

• Package Name: quarkus-langchain4j-ollama-deployment
• Group ID: io.quarkiverse.langchain4j
• Artifact ID: quarkus-langchain4j-ollama-deployment
• Package Type: Maven
• Language: Java
• Version: 1.7.4
• Installation: Add to your Quarkus project's pom.xml:

<dependency>
    <groupId>io.quarkiverse.langchain4j</groupId>
    <artifactId>quarkus-langchain4j-ollama-deployment</artifactId>
    <version>1.7.4</version>
</dependency>

Overview

This is a Quarkus deployment module that operates during the build phase of Quarkus applications. It does not provide runtime APIs for end-user applications. Instead, it:

• Performs build-time processing to configure Ollama integration
• Registers Ollama as a provider for chat and embedding models
• Creates synthetic CDI beans for runtime use
• Configures GraalVM native image compilation support
• Manages DevServices for automatic Ollama container provisioning during development
• Integrates with Quarkus Dev UI for enhanced development experience

The deployment module works in conjunction with its runtime counterpart (quarkus-langchain4j-ollama) to provide a complete solution for developers building AI-powered applications with Ollama.

Module Purpose

As a Quarkus extension deployment module, this package is used by the Quarkus Maven/Gradle plugins during the application build process. It participates in the Quarkus build chain through annotated build step methods that:

1. Register features and index dependencies
2. Configure native image reflection support
3. Register model providers (chat and embedding)
4. Determine DevServices requirements
5. Generate synthetic CDI beans at runtime initialization
6. Configure REST client serialization preferences
7. Integrate with Quarkus Dev UI

Important: While this is a deployment module that operates at build time, end users primarily interact with it through runtime configuration properties and CDI bean injection. The deployment module creates beans that are configured at runtime via application.properties.

Key Features

• Build-time Configuration: Enables/disables chat and embedding models through configuration properties
• Runtime Configuration: Comprehensive configuration for Ollama connection, model selection, and model parameters
• DevServices Integration: Automatically starts Ollama container during development and testing
• Native Image Support: Configures reflection and service provider registration for GraalVM
• Named Configurations: Supports multiple Ollama instances with different configurations
• Synthetic Bean Creation: Programmatically creates CDI beans for ChatModel, StreamingChatModel, and EmbeddingModel
• Dev UI Integration: Adds "Open WebUI" card to Quarkus Dev UI

Core Imports

Application code typically imports LangChain4j model interfaces and Quarkus CDI annotations:

// LangChain4j model interfaces
import dev.langchain4j.model.chat.ChatModel;
import dev.langchain4j.model.chat.StreamingChatModel;
import dev.langchain4j.model.embedding.EmbeddingModel;

// CDI injection
import jakarta.inject.Inject;

// Named configuration qualifier (for multiple Ollama instances)
import io.quarkiverse.langchain4j.ModelName;

For advanced use cases, you may also import:

// Chat model listeners for observability
import io.quarkiverse.langchain4j.ChatModelListener;

// Message types
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.data.message.SystemMessage;

Basic Usage

Simple Chat Model Usage

import jakarta.inject.Inject;
import dev.langchain4j.model.chat.ChatModel;

public class MyService {
    @Inject
    ChatModel chatModel;

    public String chat(String userMessage) {
        return chatModel.generate(userMessage);
    }
}

Configuration (application.properties):

# Ollama connection (optional if using DevServices)
quarkus.langchain4j.ollama.base-url=http://localhost:11434

# Model selection
quarkus.langchain4j.ollama.chat-model.model-id=llama3.2

# Model parameters
quarkus.langchain4j.ollama.chat-model.temperature=0.7

Named Configuration Usage

For multiple Ollama instances:

import jakarta.inject.Inject;
import dev.langchain4j.model.chat.ChatModel;
import io.quarkiverse.langchain4j.ModelName;

public class MyService {
    @Inject
    @ModelName("creative")
    ChatModel creativeModel;

    @Inject
    @ModelName("precise")
    ChatModel preciseModel;
}

Configuration:

# Creative instance
quarkus.langchain4j.ollama.creative.base-url=http://localhost:11434
quarkus.langchain4j.ollama.creative.chat-model.model-id=llama3.2
quarkus.langchain4j.ollama.creative.chat-model.temperature=1.2

# Precise instance
quarkus.langchain4j.ollama.precise.base-url=http://localhost:11434
quarkus.langchain4j.ollama.precise.chat-model.model-id=llama3.2
quarkus.langchain4j.ollama.precise.chat-model.temperature=0.2

Architecture

This deployment module follows the standard Quarkus extension architecture with multiple processors participating in the build chain, DevServices integration, and synthetic bean generation.

Architecture

Capabilities

Runtime Configuration

Configure Ollama connection, model selection, and model parameters at runtime.

@ConfigRoot(phase = RUN_TIME)
@ConfigMapping(prefix = "quarkus.langchain4j.ollama")
public interface LangChain4jOllamaConfig {
    OllamaConfig defaultConfig();
    Map<String, OllamaConfig> namedConfig();
}

Runtime Configuration

Runtime Model Types

Runtime type definitions and data structures used by Ollama models.

Runtime Model Types

Build-Time Configuration

Configure Ollama integration at build time through application properties.

@ConfigRoot(phase = BUILD_TIME)
@ConfigMapping(prefix = "quarkus.langchain4j.ollama")
public interface LangChain4jOllamaOpenAiBuildConfig {
    ChatModelBuildConfig chatModel();
    EmbeddingModelBuildConfig embeddingModel();
    OllamaDevServicesBuildConfig devservices();
}

Build-Time Configuration

DevServices Configuration

Configure automatic Ollama container provisioning for development environments.

@ConfigGroup
public interface OllamaDevServicesBuildConfig {
    boolean enabled();
    String imageName();
    OptionalInt port();
}

DevServices Configuration

Build Step Processing

The module participates in the Quarkus build chain through processor classes.

public class OllamaProcessor {
    @BuildStep
    FeatureBuildItem feature();

    @BuildStep
    IndexDependencyBuildItem indexUpstreamOllamaModule();

    @BuildStep
    void nativeSupport(
        BuildProducer<ServiceProviderBuildItem> serviceProviderProducer,
        BuildProducer<ReflectiveClassBuildItem> reflectiveClassProducer,
        BuildProducer<ReflectiveHierarchyBuildItem> reflectiveHierarchyProducer
    );

    @BuildStep
    void providerCandidates(
        BuildProducer<ChatModelProviderCandidateBuildItem> chatProducer,
        BuildProducer<EmbeddingModelProviderCandidateBuildItem> embeddingProducer,
        LangChain4jOllamaOpenAiBuildConfig config
    );

    @BuildStep
    void implicitlyConfiguredProviders(
        LangChain4jOllamaFixedRuntimeConfig fixedRuntimeConfig,
        BuildProducer<ImplicitlyUserConfiguredChatProviderBuildItem> producer
    );

    @BuildStep(onlyIfNot = IsNormal.class, onlyIf = Langchain4jDevServicesEnabled.class)
    void devServicesSupport(
        List<SelectedChatModelProviderBuildItem> selectedChatModels,
        List<SelectedEmbeddingModelCandidateBuildItem> selectedEmbeddingModels,
        LangChain4jOllamaFixedRuntimeConfig fixedRuntimeConfig,
        BuildProducer<DevServicesChatModelRequiredBuildItem> chatProducer,
        BuildProducer<DevServicesEmbeddingModelRequiredBuildItem> embeddingProducer
    );

    @BuildStep
    @Record(ExecutionTime.RUNTIME_INIT)
    void generateBeans(
        OllamaRecorder recorder,
        List<SelectedChatModelProviderBuildItem> selectedChatItem,
        List<SelectedEmbeddingModelCandidateBuildItem> selectedEmbedding,
        BuildProducer<SyntheticBeanBuildItem> beanProducer
    );

    @BuildStep
    void deprioritizeJsonb(
        Capabilities capabilities,
        BuildProducer<MessageBodyReaderOverrideBuildItem> readerOverrideProducer,
        BuildProducer<MessageBodyWriterOverrideBuildItem> writerOverrideProducer
    );
}

Build Step Processing

Native Image Support

Configure GraalVM native image compilation support for Ollama classes.

Native Image Configuration

Synthetic Bean Generation

Create CDI beans programmatically at runtime initialization for injection into application code.

Synthetic Bean Generation

Configuration Properties

This section provides a quick reference for common configuration properties. See Runtime Configuration for complete runtime configuration documentation and Build-Time Configuration for build-time configuration details.

Runtime Properties

These properties configure Ollama connection and model behavior at runtime (can be changed without rebuilding):

Build-Time Properties

These properties control build-time behavior and model enablement (requires rebuild to change):

Named Configuration Support

All properties support named instances using the pattern:

quarkus.langchain4j.ollama.<config-name>.*

For example:

quarkus.langchain4j.ollama.my-ollama.chat-model.enabled=true
quarkus.langchain4j.ollama.my-ollama.devservices.enabled=true

Runtime Beans Created

The deployment module creates the following synthetic CDI beans at runtime initialization:

ChatModel Bean

• Type: dev.langchain4j.model.chat.ChatModel
• Scope: @ApplicationScoped
• Qualifier: @ModelName (for named configurations)
• Injection: Includes Instance<ChatModelListener> injection point

StreamingChatModel Bean

• Type: dev.langchain4j.model.chat.StreamingChatModel
• Scope: @ApplicationScoped
• Qualifier: @ModelName (for named configurations)
• Injection: Includes Instance<ChatModelListener> injection point

EmbeddingModel Bean

• Type: dev.langchain4j.model.embedding.EmbeddingModel
• Scope: @ApplicationScoped
• Qualifier: @ModelName (for named configurations)
• Unremovable: true

DevServices Behavior

DevServices automatically starts an Ollama container during development and testing when:

1. The application is not running in production mode (IsNormal.class is false)
2. DevServices are enabled (Langchain4jDevServicesEnabled condition is true)
3. The base URL is not configured OR points to localhost
4. Docker is available and running
5. Ollama is not already running on localhost:11434

The DevServices container:

• Uses the configured image (default: ollama/ollama:latest)
• Exposes a random port (or fixed port if configured)
• Shares the local ~/.ollama directory for model caching
• Automatically shuts down when the application stops
• Can be shared across multiple applications with the label quarkus-dev-service-ollama

Build Items

Produced Build Items

The deployment module produces these build items for the Quarkus build chain:

• FeatureBuildItem - Registers features
• IndexDependencyBuildItem - Indexes upstream dependencies
• ServiceProviderBuildItem - Registers service providers
• ReflectiveClassBuildItem - Registers classes for reflection
• ReflectiveHierarchyBuildItem - Registers class hierarchies for reflection
• ChatModelProviderCandidateBuildItem - Registers chat model provider
• EmbeddingModelProviderCandidateBuildItem - Registers embedding model provider
• ImplicitlyUserConfiguredChatProviderBuildItem - Registers named configurations
• DevServicesChatModelRequiredBuildItem - Indicates chat model DevServices requirement
• DevServicesEmbeddingModelRequiredBuildItem - Indicates embedding model DevServices requirement
• DevServicesOllamaConfigBuildItem - Provides DevServices configuration
• DevServicesResultBuildItem - Provides DevServices lifecycle information
• SyntheticBeanBuildItem - Creates synthetic CDI beans
• MessageBodyReaderOverrideBuildItem - Overrides reader priority
• MessageBodyWriterOverrideBuildItem - Overrides writer priority
• AdditionalDevUiCardBuildItem - Adds Dev UI card

Consumed Build Items

The deployment module consumes these build items:

• DockerStatusBuildItem - Docker availability
• LaunchModeBuildItem - Application launch mode
• ConsoleInstalledBuildItem - Console availability
• LoggingSetupBuildItem - Logging configuration
• DevServicesSharedNetworkBuildItem - Shared network configuration
• SelectedChatModelProviderBuildItem - Selected chat model provider
• SelectedEmbeddingModelCandidateBuildItem - Selected embedding model provider
• Capabilities - Quarkus capabilities

Package Types

Configuration Interfaces

// Root build-time configuration
@ConfigRoot(phase = BUILD_TIME)
@ConfigMapping(prefix = "quarkus.langchain4j.ollama")
public interface LangChain4jOllamaOpenAiBuildConfig {
    ChatModelBuildConfig chatModel();
    EmbeddingModelBuildConfig embeddingModel();
    OllamaDevServicesBuildConfig devservices();
}

// Chat model build configuration
@ConfigGroup
public interface ChatModelBuildConfig {
    @ConfigDocDefault("true")
    Optional<Boolean> enabled();
}

// Embedding model build configuration
@ConfigGroup
public interface EmbeddingModelBuildConfig {
    @ConfigDocDefault("true")
    Optional<Boolean> enabled();
}

// DevServices configuration
@ConfigGroup
public interface OllamaDevServicesBuildConfig {
    String OLLAMA_IMAGE = "ollama/ollama:latest";

    @WithDefault("true")
    boolean enabled();

    @WithDefault(OLLAMA_IMAGE)
    String imageName();

    OptionalInt port();
}

Module Dependencies

Compile Dependencies

• io.quarkus:quarkus-arc-deployment - CDI container deployment support
• io.quarkus:quarkus-rest-client-jackson-deployment - REST client Jackson support
• io.quarkus:quarkus-qute-deployment - Qute templating support
• io.quarkiverse.langchain4j:quarkus-langchain4j-core-deployment - Core LangChain4j deployment
• io.quarkiverse.langchain4j:quarkus-langchain4j-ollama - Ollama runtime module
• org.testcontainers:ollama:1.21.3 - Testcontainers Ollama support

Usage Context

This deployment module is typically added as a dependency in a Quarkus application's pom.xml when using the Ollama extension. Application developers interact with the runtime beans (ChatModel, StreamingChatModel, EmbeddingModel) created by this module, not with the deployment module directly.

Example application dependency:

<dependency>
    <groupId>io.quarkiverse.langchain4j</groupId>
    <artifactId>quarkus-langchain4j-ollama</artifactId>
</dependency>

The deployment module is automatically included as a transitive dependency and participates in the Quarkus build process.

Notes

• This module operates exclusively at build time and does not provide runtime APIs for application code
• Configuration is done through standard Quarkus configuration properties in application.properties or application.yml
• The module creates synthetic CDI beans that applications inject and use at runtime
• DevServices provide automatic Ollama provisioning for development without manual setup
• Supports GraalVM native image compilation with proper reflection configuration
• Uses Jackson for JSON serialization (deprioritizes JSON-B when both present)
• Integrates with the Quarkus provider selection mechanism for flexible model configuration