CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/maven-dev-langchain4j--langchain4j-github-models

This package provides a deprecated integration module that enables Java applications to interact with GitHub Models through the LangChain4j framework. It offers chat models (both synchronous and streaming), embedding models, and support for AI services with tool integration, JSON schema responses, and responsible AI features. The module wraps Azure AI Inference SDK to provide a unified API for accessing various language models hosted on GitHub Models, including chat completion capabilities, embeddings generation, and content filtering management. As of version 1.10.0, this module has been marked for deprecation and future removal, with users recommended to migrate to the langchain4j-openai-official module for enhanced functionality and better integration. The library is designed for reusability as a foundational component in LLM-powered Java applications that need to leverage GitHub-hosted AI models, offering builder patterns for configuration, support for proxy options, custom timeouts, and comprehensive model service versioning capabilities.

—

Pending

Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

Overview
Eval results
Files

chat-model-guide.mddocs/guides/

Chat Model Usage Guide

Practical guide for using GitHubModelsChatModel with examples and patterns.

Basic Chat Completion

Simple question answering

GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .gitHubToken(System.getenv("GITHUB_TOKEN"))
    .modelName(GitHubModelsChatModelName.GPT_4_O)
    .build();

ChatRequest request = ChatRequest.builder()
    .messages(UserMessage.from("What is the capital of France?"))
    .build();

ChatResponse response = model.chat(request);
System.out.println(response.aiMessage().text());

With system message

ChatRequest request = ChatRequest.builder()
    .messages(
        SystemMessage.from("You are a helpful assistant specialized in geography."),
        UserMessage.from("Tell me about Paris.")
    )
    .build();

ChatResponse response = model.chat(request);

Multi-turn conversation

List<ChatMessage> conversation = new ArrayList<>();
conversation.add(SystemMessage.from("You are a helpful assistant."));
conversation.add(UserMessage.from("Hello!"));

ChatResponse response1 = model.chat(ChatRequest.builder()
    .messages(conversation)
    .build());

conversation.add(response1.aiMessage());
conversation.add(UserMessage.from("Tell me a joke."));

ChatResponse response2 = model.chat(ChatRequest.builder()
    .messages(conversation)
    .build());

Tool Calling

Define and use tools

// Define tool
ToolSpecification weatherTool = ToolSpecification.builder()
    .name("get_weather")
    .description("Get current weather for a location")
    .addParameter("location", "string", "The city name")
    .addParameter("unit", "string", "Temperature unit (celsius/fahrenheit)")
    .build();

// Create request with tool
ChatRequest request = ChatRequest.builder()
    .messages(UserMessage.from("What's the weather in Paris?"))
    .parameters(ChatRequestParameters.builder()
        .toolSpecifications(weatherTool)
        .build())
    .build();

ChatResponse response = model.chat(request);

// Check if tool was called
if (response.aiMessage().hasToolExecutionRequests()) {
    for (ToolExecutionRequest toolRequest : response.aiMessage().toolExecutionRequests()) {
        String toolName = toolRequest.name();
        String arguments = toolRequest.arguments();

        // Execute tool
        String result = executeWeatherTool(toolName, arguments);

        // Continue conversation with tool result
        List<ChatMessage> messages = new ArrayList<>();
        messages.add(UserMessage.from("What's the weather in Paris?"));
        messages.add(response.aiMessage());
        messages.add(ToolExecutionResultMessage.from(toolRequest, result));

        ChatResponse finalResponse = model.chat(ChatRequest.builder()
            .messages(messages)
            .build());

        System.out.println(finalResponse.aiMessage().text());
    }
}

Multiple tools

ToolSpecification weatherTool = ToolSpecification.builder()
    .name("get_weather")
    .description("Get current weather")
    .addParameter("location", "string", "City name")
    .build();

ToolSpecification timeTool = ToolSpecification.builder()
    .name("get_time")
    .description("Get current time")
    .addParameter("timezone", "string", "Timezone")
    .build();

ChatRequest request = ChatRequest.builder()
    .messages(UserMessage.from("What's the weather and time in Paris?"))
    .parameters(ChatRequestParameters.builder()
        .toolSpecifications(weatherTool, timeTool)
        .build())
    .build();

ChatResponse response = model.chat(request);

Note: When using ToolChoice.REQUIRED, only a single tool is supported.

JSON Output

JSON object response

import com.azure.ai.inference.models.ChatCompletionsResponseFormatJsonObject;

GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .gitHubToken(token)
    .modelName("gpt-4o")
    .responseFormat(new ChatCompletionsResponseFormatJsonObject())
    .build();

ChatRequest request = ChatRequest.builder()
    .messages(
        SystemMessage.from("You are a helpful assistant that outputs JSON."),
        UserMessage.from("List three countries with their capitals in JSON format.")
    )
    .build();

ChatResponse response = model.chat(request);
String jsonText = response.aiMessage().text();
// Parse jsonText as needed

Strict JSON schema

GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .gitHubToken(token)
    .modelName("gpt-4o")
    .responseFormat(new ChatCompletionsResponseFormatJsonObject())
    .strictJsonSchema(true)
    .build();

Sampling Parameters

Control randomness and creativity

// More creative (higher temperature)
GitHubModelsChatModel creativeModel = GitHubModelsChatModel.builder()
    .gitHubToken(token)
    .modelName("gpt-4o")
    .temperature(1.2)
    .topP(0.95)
    .build();

// More deterministic (lower temperature)
GitHubModelsChatModel deterministicModel = GitHubModelsChatModel.builder()
    .gitHubToken(token)
    .modelName("gpt-4o")
    .temperature(0.2)
    .topP(0.9)
    .build();

Limit output length

GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .gitHubToken(token)
    .modelName("gpt-4o")
    .maxTokens(100)  // Limit to 100 tokens
    .build();

Encourage variety (penalties)

GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .gitHubToken(token)
    .modelName("gpt-4o")
    .presencePenalty(0.6)   // Encourage new topics
    .frequencyPenalty(0.3)  // Reduce repetition
    .build();

Deterministic generation

GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .gitHubToken(token)
    .modelName("gpt-4o")
    .seed(12345L)  // Same seed produces similar outputs
    .temperature(0.0)
    .build();

// Calling multiple times with same input produces consistent results

Stop sequences

GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .gitHubToken(token)
    .modelName("gpt-4o")
    .stop(Arrays.asList("\n\n", "END", "---"))
    .build();

// Model stops generating when it encounters these sequences

Metadata and Token Usage

Check token consumption

ChatResponse response = model.chat(request);

TokenUsage usage = response.metadata().tokenUsage();
System.out.println("Input tokens: " + usage.inputTokenCount());
System.out.println("Output tokens: " + usage.outputTokenCount());
System.out.println("Total tokens: " + usage.totalTokenCount());

Check finish reason

ChatResponse response = model.chat(request);

FinishReason reason = response.metadata().finishReason();
switch (reason) {
    case STOP:
        System.out.println("Natural completion");
        break;
    case LENGTH:
        System.out.println("Stopped due to max_tokens");
        break;
    case TOOL_EXECUTION:
        System.out.println("Stopped to execute tool");
        break;
    case CONTENT_FILTER:
        System.out.println("Content filtered");
        break;
}

Model Selection Patterns

Choose by task

// Fast responses, simple tasks
.modelName(GitHubModelsChatModelName.GPT_4_O_MINI)

// Complex reasoning, high quality
.modelName(GitHubModelsChatModelName.GPT_4_O)

// Vision tasks
.modelName(GitHubModelsChatModelName.PHI_3_5_VISION_INSTRUCT)

// Long context (128K tokens)
.modelName(GitHubModelsChatModelName.PHI_3_MEDIUM_INSTRUCT_128K)

Open source models

// Small, fast
.modelName(GitHubModelsChatModelName.META_LLAMA_3_1_8B_INSTRUCT)

// Balanced
.modelName(GitHubModelsChatModelName.META_LLAMA_3_1_70B_INSTRUCT)

// Maximum capability
.modelName(GitHubModelsChatModelName.META_LLAMA_3_1_405B_INSTRUCT)

Capabilities Check

Query supported features

Set<Capability> capabilities = model.supportedCapabilities();

if (capabilities.contains(Capability.RESPONSE_FORMAT_JSON_SCHEMA)) {
    // Model supports JSON schema responses
}

if (capabilities.contains(Capability.TOOL_CALLING)) {
    // Model supports tool calling
}

Common Patterns

Question answering with context

String context = loadDocumentContext();

ChatRequest request = ChatRequest.builder()
    .messages(
        SystemMessage.from("Answer questions based on this context: " + context),
        UserMessage.from("What is the main topic?")
    )
    .build();

ChatResponse response = model.chat(request);

Classification task

GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .gitHubToken(token)
    .modelName("gpt-4o-mini")
    .temperature(0.0)  // Deterministic for classification
    .maxTokens(10)     // Short response
    .build();

ChatRequest request = ChatRequest.builder()
    .messages(
        SystemMessage.from("Classify sentiment as: positive, negative, or neutral"),
        UserMessage.from("I love this product!")
    )
    .build();

ChatResponse response = model.chat(request);
String sentiment = response.aiMessage().text().trim().toLowerCase();

Translation

ChatRequest request = ChatRequest.builder()
    .messages(
        SystemMessage.from("You are a translator. Translate to French."),
        UserMessage.from("Hello, how are you?")
    )
    .build();

ChatResponse response = model.chat(request);
String translation = response.aiMessage().text();

Summarization

String longText = loadLongDocument();

GitHubModelsChatModel model = GitHubModelsChatModel.builder()
    .gitHubToken(token)
    .modelName("gpt-4o")
    .maxTokens(150)
    .build();

ChatRequest request = ChatRequest.builder()
    .messages(
        SystemMessage.from("Summarize the following text in 2-3 sentences."),
        UserMessage.from(longText)
    )
    .build();

ChatResponse response = model.chat(request);
String summary = response.aiMessage().text();

See Also

  • Chat Model API Reference
  • Configuration Guide
  • Error Handling

Install with Tessl CLI

npx tessl i tessl/maven-dev-langchain4j--langchain4j-github-models@1.11.0

docs

guides

chat-model-guide.md

embedding-model-guide.md

spi-factories-guide.md

streaming-chat-model-guide.md

index.md

quick-reference.md

tile.json