CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/maven-dev-langchain4j--langchain4j-github-models

This package provides a deprecated integration module that enables Java applications to interact with GitHub Models through the LangChain4j framework. It offers chat models (both synchronous and streaming), embedding models, and support for AI services with tool integration, JSON schema responses, and responsible AI features. The module wraps Azure AI Inference SDK to provide a unified API for accessing various language models hosted on GitHub Models, including chat completion capabilities, embeddings generation, and content filtering management. As of version 1.10.0, this module has been marked for deprecation and future removal, with users recommended to migrate to the langchain4j-openai-official module for enhanced functionality and better integration. The library is designed for reusability as a foundational component in LLM-powered Java applications that need to leverage GitHub-hosted AI models, offering builder patterns for configuration, support for proxy options, custom timeouts, and comprehensive model service versioning capabilities.

—

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Overview
Eval results
Files

advanced-configuration.mddocs/configuration/

Advanced Configuration

Advanced features including custom clients, listeners, capabilities, and logging.

Custom Azure AI Clients

Custom Chat Completions Client

.chatCompletionsClient(ChatCompletionsClient chatCompletionsClient)

Provide custom Azure AI Inference client for maximum control.

When to use:

  • Custom authentication mechanisms
  • Advanced HTTP client configuration
  • Integration with existing Azure SDK setup
  • Special logging or monitoring requirements

Example:

import com.azure.ai.inference.ChatCompletionsClient;
import com.azure.ai.inference.ChatCompletionsClientBuilder;
import com.azure.core.credential.AzureKeyCredential;

ChatCompletionsClient customClient = new ChatCompletionsClientBuilder()
    .endpoint("https://models.inference.ai.azure.com")
    .credential(new AzureKeyCredential(token))
    .buildClient();

GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .chatCompletionsClient(customClient)
    .modelName("gpt-4o")
    .build();

Note: When custom client is set, most other builder options are ignored (endpoint, timeout, proxy, etc.)

Custom Async Client (Streaming)

.chatCompletionsAsyncClient(ChatCompletionsAsyncClient client)

Provide custom async client for streaming models.

import com.azure.ai.inference.ChatCompletionsAsyncClient;
import com.azure.ai.inference.ChatCompletionsAsyncClientBuilder;

ChatCompletionsAsyncClient asyncClient = new ChatCompletionsAsyncClientBuilder()
    .endpoint("https://models.inference.ai.azure.com")
    .credential(new AzureKeyCredential(token))
    .buildAsyncClient();

GitHubModelsStreamingChatModel model = GitHubModelsStreamingChatModel.builder()
    .chatCompletionsAsyncClient(asyncClient)
    .modelName("gpt-4o")
    .build();

Custom Embeddings Client

.embeddingsClient(EmbeddingsClient embeddingsClient)

Provide custom embeddings client.

import com.azure.ai.inference.EmbeddingsClient;
import com.azure.ai.inference.EmbeddingsClientBuilder;

EmbeddingsClient customClient = new EmbeddingsClientBuilder()
    .endpoint("https://models.inference.ai.azure.com")
    .credential(new AzureKeyCredential(token))
    .buildClient();

GitHubModelsEmbeddingModel model = GitHubModelsEmbeddingModel.builder()
    .embeddingsClient(customClient)
    .modelName("text-embedding-3-small")
    .build();

Model Listeners

Chat Model Listeners

.listeners(List<ChatModelListener> listeners)

Register listeners to observe model requests, responses, and errors.

Use cases:

  • Logging and auditing
  • Metrics collection
  • Request/response tracking
  • Error monitoring
  • Performance measurement

Example:

import dev.langchain4j.model.chat.listener.ChatModelListener;
import dev.langchain4j.model.chat.request.ChatRequest;
import dev.langchain4j.model.chat.response.ChatResponse;

public class LoggingListener implements ChatModelListener {
    @Override
    public void onRequest(ChatModelRequestContext context) {
        System.out.println("Request: " + context.request());
        System.out.println("Model: " + context.model());
    }

    @Override
    public void onResponse(ChatModelResponseContext context) {
        ChatResponse response = context.response();
        System.out.println("Response: " + response.aiMessage().text());
        System.out.println("Tokens: " + response.metadata().tokenUsage().totalTokenCount());
    }

    @Override
    public void onError(ChatModelErrorContext context) {
        System.err.println("Error: " + context.error().getMessage());
    }
}

GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .gitHubToken(token)
    .modelName("gpt-4o")
    .listeners(Arrays.asList(new LoggingListener()))
    .build();

Metrics Collection Listener

public class MetricsListener implements ChatModelListener {
    private final MetricsRegistry metrics;

    public MetricsListener(MetricsRegistry metrics) {
        this.metrics = metrics;
    }

    @Override
    public void onRequest(ChatModelRequestContext context) {
        metrics.increment("chat.requests");
    }

    @Override
    public void onResponse(ChatModelResponseContext context) {
        metrics.increment("chat.responses");
        int tokens = context.response().metadata().tokenUsage().totalTokenCount();
        metrics.gauge("chat.tokens", tokens);
    }

    @Override
    public void onError(ChatModelErrorContext context) {
        metrics.increment("chat.errors");
    }
}

Multiple Listeners

List<ChatModelListener> listeners = Arrays.asList(
    new LoggingListener(),
    new MetricsListener(metricsRegistry),
    new AuditListener(auditService),
    new TracingListener(tracingContext)
);

GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .gitHubToken(token)
    .modelName("gpt-4o")
    .listeners(listeners)
    .build();

Request/Response Logging

Enable Built-in Logging

.logRequestsAndResponses(Boolean logRequestsAndResponses)
// or
.logRequestsAndResponses(boolean logRequestsAndResponses)

Enable detailed logging of API requests and responses.

Use for:

  • Development and debugging
  • Issue investigation
  • Understanding API interactions

Example:

GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .gitHubToken(token)
    .modelName("gpt-4o")
    .logRequestsAndResponses(true)
    .build();

Warning: Logs may contain sensitive data. Don't enable in production unless logs are secured.

Conditional Logging

boolean isDebugMode = "true".equals(System.getenv("DEBUG"));

GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .gitHubToken(token)
    .modelName("gpt-4o")
    .logRequestsAndResponses(isDebugMode)
    .build();

Environment-Based Logging

boolean shouldLog = !"production".equals(System.getenv("APP_ENV"));

GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .gitHubToken(token)
    .modelName("gpt-4o")
    .logRequestsAndResponses(shouldLog)
    .build();

Supported Capabilities

Override Supported Capabilities

.supportedCapabilities(Set<Capability> supportedCapabilities)

Explicitly set the capabilities for this model instance (chat model only).

Example:

import dev.langchain4j.model.chat.Capability;

Set<Capability> capabilities = new HashSet<>(Arrays.asList(
    Capability.RESPONSE_FORMAT_JSON_SCHEMA,
    Capability.TOOL_CALLING
));

GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .gitHubToken(token)
    .modelName("gpt-4o")
    .supportedCapabilities(capabilities)
    .build();

Note: Typically not needed - models auto-detect capabilities. Only use when you need to explicitly limit or override detected capabilities.

Complete Advanced Configuration Example

Production-Ready Configuration

import com.azure.ai.inference.ChatCompletionsClient;
import com.azure.ai.inference.ChatCompletionsClientBuilder;
import com.azure.core.credential.AzureKeyCredential;
import com.azure.core.http.ProxyOptions;
import dev.langchain4j.model.chat.listener.ChatModelListener;
import java.time.Duration;
import java.net.InetSocketAddress;
import java.util.*;

public class AdvancedModelConfig {

    public static GitHubModelsChatModel createProductionChatModel() {
        // Metrics and monitoring
        List<ChatModelListener> listeners = Arrays.asList(
            new MetricsListener(),
            new AuditListener(),
            new TracingListener()
        );

        // Custom headers for tracking
        Map<String, String> headers = new HashMap<>();
        headers.put("X-Application", "production-service");
        headers.put("X-Version", getAppVersion());
        headers.put("X-Environment", "production");

        // Proxy configuration
        ProxyOptions proxy = null;
        if (System.getenv("HTTP_PROXY_HOST") != null) {
            proxy = new ProxyOptions(
                ProxyOptions.Type.HTTP,
                new InetSocketAddress(
                    System.getenv("HTTP_PROXY_HOST"),
                    Integer.parseInt(System.getenv("HTTP_PROXY_PORT", "8080"))
                )
            );
        }

        return GitHubModelsChatModel.builder()
            .gitHubToken(loadSecureToken())
            .modelName(GitHubModelsChatModelName.GPT_4_O)
            .temperature(0.7)
            .maxTokens(2000)
            .timeout(Duration.ofSeconds(60))
            .maxRetries(5)
            .proxyOptions(proxy)
            .customHeaders(headers)
            .listeners(listeners)
            .userAgentSuffix("prod-service/2.1.0")
            .logRequestsAndResponses(false)  // Disabled in production
            .build();
    }

    private static String loadSecureToken() {
        // Load from secure source
        return System.getenv("GITHUB_TOKEN");
    }

    private static String getAppVersion() {
        return "2.1.0";
    }
}

Development Configuration with Custom Client

public class DevelopmentModelConfig {

    public static GitHubModelsChatModel createDevChatModel() {
        // Custom client with detailed logging
        ChatCompletionsClient customClient = new ChatCompletionsClientBuilder()
            .endpoint("https://models.inference.ai.azure.com")
            .credential(new AzureKeyCredential(System.getenv("GITHUB_TOKEN")))
            .addPolicy(new DetailedLoggingPolicy())
            .buildClient();

        // Development listeners
        List<ChatModelListener> listeners = Arrays.asList(
            new ConsoleLoggingListener(),
            new DebugMetricsListener()
        );

        return GitHubModelsChatModel.builder()
            .chatCompletionsClient(customClient)
            .modelName("gpt-4o")
            .listeners(listeners)
            .logRequestsAndResponses(true)
            .build();
    }
}

Testing Configuration

Test with Mock Client

@Test
public void testChatModel() {
    ChatCompletionsClient mockClient = mock(ChatCompletionsClient.class);

    // Configure mock
    when(mockClient.complete(any())).thenReturn(createMockResponse());

    GitHubModelsChatModel model = GitHubModelsChatModel.builder()
        .chatCompletionsClient(mockClient)
        .modelName("test-model")
        .build();

    // Test
    ChatResponse response = model.chat(request);

    // Verify
    verify(mockClient).complete(any());
}

Test with Listeners

@Test
public void testWithListeners() {
    TestListener testListener = new TestListener();

    GitHubModelsChatModel model = GitHubModelsChatModel.builder()
        .gitHubToken("test-token")
        .modelName("test-model")
        .listeners(Arrays.asList(testListener))
        .build();

    model.chat(request);

    assertTrue(testListener.wasRequestCalled());
    assertTrue(testListener.wasResponseCalled());
}

Best Practices

Use Custom Clients Sparingly

// ✅ Good - use builder configuration
GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .gitHubToken(token)
    .modelName("gpt-4o")
    .timeout(Duration.ofSeconds(60))
    .build();

// ⚠️ Only when necessary - custom client
GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .chatCompletionsClient(customClient)
    .modelName("gpt-4o")
    .build();

Add Meaningful Listeners

// ✅ Good - purposeful listeners
.listeners(Arrays.asList(
    new MetricsListener(metrics),
    new AuditListener(audit)
))

// ❌ Bad - empty/no-op listeners
.listeners(Arrays.asList(new NoOpListener()))

Disable Logging in Production

// ✅ Good - conditional logging
boolean isProduction = "prod".equals(System.getenv("ENV"));
.logRequestsAndResponses(!isProduction)

// ❌ Bad - always enabled
.logRequestsAndResponses(true)

Document Custom Configurations

/**
 * Creates chat model with custom configuration for high-volume processing:
 * - Uses custom Azure client with connection pooling
 * - Metrics listener for monitoring
 * - 90-second timeout for complex requests
 * - 5 retries for resilience
 */
public static GitHubModelsChatModel createBatchProcessingModel() {
    // Implementation
}

See Also

  • Builder Configuration
  • Network Configuration
  • Chat Model API
  • Error Handling

Install with Tessl CLI

npx tessl i tessl/maven-dev-langchain4j--langchain4j-github-models@1.11.0

docs

configuration

advanced-configuration.md

authentication.md

builder-configuration.md

network-configuration.md

index.md

quick-reference.md

tile.json