tessl/maven-dev-langchain4j--langchain4j-google-ai-gemini

LangChain4j integration for Google AI Gemini models providing chat, streaming, embeddings, image generation, and batch processing capabilities

Overview

Eval results

Files

Batch Processing

Name: tessl/maven-dev-langchain4j--langchain4j-google-ai-gemini
Author: tessl

Cost-efficient batch processing operations offering 50% cost reduction compared to synchronous requests, with a 24-hour processing Service Level Objective (SLO). Batch processing supports chat requests, embeddings, and image generation (all experimental features).

Capabilities

GoogleAiGeminiBatchChatModel

Batch processing for chat requests enabling cost-efficient processing of large volumes of requests.

/**
 * Batch chat model for cost-efficient processing of multiple chat requests.
 * Offers 50% cost reduction with 24-hour processing SLO.
 */
public class GoogleAiGeminiBatchChatModel {
    /**
     * Creates a new builder for configuring the batch chat model.
     * @return Builder instance
     */
    public static Builder builder();

    /**
     * Creates a batch job from inline requests.
     * @param displayName Human-readable name for the batch job
     * @param priority Priority level (optional, higher values = higher priority)
     * @param requests List of chat requests to process
     * @return BatchResponse with job status or results
     */
    public BatchResponse<ChatResponse> createBatchInline(
        String displayName,
        Long priority,
        List<ChatRequest> requests
    );

    /**
     * Creates a batch job from a previously uploaded file.
     * @param displayName Human-readable name for the batch job
     * @param file GeminiFile containing batch requests in JSON Lines format
     * @return BatchResponse with job status or results
     */
    public BatchResponse<ChatResponse> createBatchFromFile(
        String displayName,
        GeminiFile file
    );

    /**
     * Writes batch requests to a file in JSON Lines format.
     * @param writer JsonLinesWriter for writing to file
     * @param requests Iterable of batch file requests with keys
     * @throws IOException if an I/O error occurs while writing
     */
    public void writeBatchToFile(
        JsonLinesWriter writer,
        Iterable<BatchFileRequest<ChatRequest>> requests
    ) throws IOException;

    /**
     * Retrieves results from a batch job.
     * @param name Batch job identifier
     * @return BatchResponse with results or current status
     */
    public BatchResponse<ChatResponse> retrieveBatchResults(BatchName name);

    /**
     * Cancels a running batch job.
     * @param name Batch job identifier
     */
    public void cancelBatchJob(BatchName name);

    /**
     * Deletes a batch job and its results.
     * @param name Batch job identifier
     */
    public void deleteBatchJob(BatchName name);

    /**
     * Lists all batch jobs with optional pagination.
     * @param pageSize Maximum number of results per page (optional)
     * @param pageToken Token for next page from previous call (optional)
     * @return BatchList containing batch jobs and next page token
     */
    public BatchList<ChatResponse> listBatchJobs(Integer pageSize, String pageToken);
}

GoogleAiGeminiBatchEmbeddingModel

Batch processing for embedding generation.

/**
 * Batch embedding model for cost-efficient embedding generation.
 */
public class GoogleAiGeminiBatchEmbeddingModel {
    /**
     * Creates a new builder for configuring the batch embedding model.
     * @return Builder instance
     */
    public static Builder builder();

    /**
     * Creates a batch job from inline text segments.
     * @param displayName Human-readable name for the batch job
     * @param priority Priority level (optional)
     * @param segments List of text segments to embed
     * @return BatchResponse with job status or results
     */
    public BatchResponse<Embedding> createBatchInline(
        String displayName,
        Long priority,
        List<TextSegment> segments
    );

    /**
     * Creates a batch job from a previously uploaded file.
     * @param displayName Human-readable name for the batch job
     * @param file GeminiFile containing batch requests
     * @return BatchResponse with job status or results
     */
    public BatchResponse<Embedding> createBatchFromFile(
        String displayName,
        GeminiFile file
    );

    /**
     * Writes batch requests to a file in JSON Lines format.
     * @param writer JsonLinesWriter for writing to file
     * @param requests Iterable of batch file requests
     * @throws IOException if an I/O error occurs while writing
     */
    public void writeBatchToFile(
        JsonLinesWriter writer,
        Iterable<BatchFileRequest<TextSegment>> requests
    ) throws IOException;

    /**
     * Retrieves results from a batch job.
     * @param name Batch job identifier
     * @return BatchResponse with results or current status
     */
    public BatchResponse<Embedding> retrieveBatchResults(BatchName name);

    /**
     * Cancels a running batch job.
     * @param name Batch job identifier
     */
    public void cancelBatchJob(BatchName name);

    /**
     * Deletes a batch job and its results.
     * @param name Batch job identifier
     */
    public void deleteBatchJob(BatchName name);

    /**
     * Lists all batch jobs with optional pagination.
     * @param pageSize Maximum number of results per page (optional)
     * @param pageToken Token for next page (optional)
     * @return BatchList containing batch jobs
     */
    public BatchList<Embedding> listBatchJobs(Integer pageSize, String pageToken);
}

GoogleAiGeminiBatchImageModel

Batch processing for image generation requests.

/**
 * Batch image model for cost-efficient image generation.
 */
public class GoogleAiGeminiBatchImageModel {
    /**
     * Creates a new builder for configuring the batch image model.
     * @return GoogleAiGeminiBatchImageModelBuilder instance
     */
    public static GoogleAiGeminiBatchImageModelBuilder builder();

    /**
     * Creates a batch job from inline image generation requests.
     * @param displayName Human-readable name for the batch job
     * @param priority Priority level (optional)
     * @param requests List of image generation requests
     * @return BatchResponse with job status or results
     */
    public BatchResponse<Response<Image>> createBatchInline(
        String displayName,
        Long priority,
        List<ImageGenerationRequest> requests
    );

    /**
     * Creates a batch job from a previously uploaded file.
     * @param displayName Human-readable name for the batch job
     * @param file GeminiFile containing batch requests
     * @return BatchResponse with job status or results
     */
    public BatchResponse<Response<Image>> createBatchFromFile(
        String displayName,
        GeminiFile file
    );

    /**
     * Writes batch requests to a file in JSON Lines format.
     * @param writer JsonLinesWriter for writing to file
     * @param requests Iterable of batch file requests
     * @throws IOException if an I/O error occurs while writing
     */
    public void writeBatchToFile(
        JsonLinesWriter writer,
        Iterable<BatchFileRequest<ImageGenerationRequest>> requests
    ) throws IOException;

    /**
     * Retrieves results from a batch job.
     * @param name Batch job identifier
     * @return BatchResponse with results or current status
     */
    public BatchResponse<Response<Image>> retrieveBatchResults(BatchName name);

    /**
     * Cancels a running batch job.
     * @param name Batch job identifier
     */
    public void cancelBatchJob(BatchName name);

    /**
     * Deletes a batch job and its results.
     * @param name Batch job identifier
     */
    public void deleteBatchJob(BatchName name);

    /**
     * Lists all batch jobs with optional pagination.
     * @param pageSize Maximum number of results per page (optional)
     * @param pageToken Token for next page (optional)
     * @return BatchList containing batch jobs
     */
    public BatchList<Response<Image>> listBatchJobs(Integer pageSize, String pageToken);
}

/**
 * Image generation request for batch processing.
 */
public record ImageGenerationRequest(String prompt) {}

GoogleAiGeminiBatchImageModelBuilder

Builder class for constructing GoogleAiGeminiBatchImageModel with configuration options.

/**
 * Builder for GoogleAiGeminiBatchImageModel.
 */
public static class GoogleAiGeminiBatchImageModelBuilder {
    /**
     * Sets the HTTP client builder for customizing requests.
     * @param httpClientBuilder HTTP client builder instance
     * @return Builder instance for chaining
     */
    public GoogleAiGeminiBatchImageModelBuilder httpClientBuilder(HttpClientBuilder httpClientBuilder);

    /**
     * Sets the API key for authentication (required).
     * @param apiKey Google AI API key
     * @return Builder instance for chaining
     */
    public GoogleAiGeminiBatchImageModelBuilder apiKey(String apiKey);

    /**
     * Sets the base URL for the API endpoint.
     * @param baseUrl Custom base URL (optional)
     * @return Builder instance for chaining
     */
    public GoogleAiGeminiBatchImageModelBuilder baseUrl(String baseUrl);

    /**
     * Sets the model name to use for image generation.
     * @param modelName Model identifier (e.g., "gemini-2.5-flash")
     * @return Builder instance for chaining
     */
    public GoogleAiGeminiBatchImageModelBuilder modelName(String modelName);

    /**
     * Sets the aspect ratio for generated images.
     * @param aspectRatio Aspect ratio (e.g., "1:1", "16:9", "4:3")
     * @return Builder instance for chaining
     */
    public GoogleAiGeminiBatchImageModelBuilder aspectRatio(String aspectRatio);

    /**
     * Sets the size specification for generated images.
     * @param imageSize Image size (e.g., "256x256", "512x512", "1024x1024")
     * @return Builder instance for chaining
     */
    public GoogleAiGeminiBatchImageModelBuilder imageSize(String imageSize);

    /**
     * Sets the request timeout duration.
     * @param timeout Timeout duration
     * @return Builder instance for chaining
     */
    public GoogleAiGeminiBatchImageModelBuilder timeout(Duration timeout);

    /**
     * Enables logging of both requests and responses.
     * @param logRequestsAndResponses True to enable full logging
     * @return Builder instance for chaining
     */
    public GoogleAiGeminiBatchImageModelBuilder logRequestsAndResponses(Boolean logRequestsAndResponses);

    /**
     * Enables logging of requests only.
     * @param logRequests True to enable request logging
     * @return Builder instance for chaining
     */
    public GoogleAiGeminiBatchImageModelBuilder logRequests(Boolean logRequests);

    /**
     * Enables logging of responses only.
     * @param logResponses True to enable response logging
     * @return Builder instance for chaining
     */
    public GoogleAiGeminiBatchImageModelBuilder logResponses(Boolean logResponses);

    /**
     * Sets a custom logger for the model.
     * @param logger Logger instance
     * @return Builder instance for chaining
     */
    public GoogleAiGeminiBatchImageModelBuilder logger(Logger logger);

    /**
     * Sets safety settings for content filtering.
     * @param safetySettings List of safety settings
     * @return Builder instance for chaining
     */
    public GoogleAiGeminiBatchImageModelBuilder safetySettings(List<GeminiSafetySetting> safetySettings);

    /**
     * Builds the GoogleAiGeminiBatchImageModel instance.
     * @return Configured GoogleAiGeminiBatchImageModel
     * @throws IllegalArgumentException if required fields are missing
     */
    public GoogleAiGeminiBatchImageModel build();
}

Batch Response Types

BatchResponse Sealed Interface

Sealed interface representing the response from batch operations.

/**
 * Response from batch operations. Can be incomplete, successful, or error.
 * @param <T> The type of results (ChatResponse, Embedding, or Response<Image>)
 */
public sealed interface BatchResponse<T>
    permits BatchIncomplete, BatchSuccess, BatchError {
}

/**
 * Batch job is still processing.
 * @param batchName Identifier for the batch job
 * @param state Current state of the batch job
 */
public record BatchIncomplete<T>(
    BatchName batchName,
    BatchJobState state
) implements BatchResponse<T> {}

/**
 * Batch job completed successfully.
 * @param batchName Identifier for the batch job
 * @param responses List of successful responses
 * @param errors List of error statuses for failed individual requests (optional)
 */
public record BatchSuccess<T>(
    BatchName batchName,
    List<T> responses,
    List<Operation.Status> errors
) implements BatchResponse<T> {}

/**
 * Batch job failed.
 * @param batchName Identifier for the batch job
 * @param code Error code
 * @param message Error message
 * @param state Final state of the batch job
 * @param details Additional error details
 */
public record BatchError<T>(
    BatchName batchName,
    int code,
    String message,
    BatchJobState state,
    List<Map<String, Object>> details
) implements BatchResponse<T> {}

Supporting Types

/**
 * List of batch jobs with pagination support.
 * @param pageToken Token for retrieving next page (null if no more pages)
 * @param responses List of batch responses
 */
public record BatchList<T>(
    String pageToken,
    List<BatchResponse<T>> responses
) {}

/**
 * Identifier for a batch job.
 * @param value The batch job name/ID
 */
public record BatchName(String value) {}

/**
 * Request wrapper for batch file operations.
 * @param key Unique key identifying this request
 * @param request The actual request object
 */
public record BatchFileRequest<REQ>(
    String key,
    REQ request
) {}

/**
 * Long-running operation status.
 * @param name Operation name
 * @param metadata Operation metadata
 * @param done Whether operation is complete
 * @param error Error status if failed
 * @param response Response if successful
 */
public record Operation<RESP>(
    String name,
    Map<String, Object> metadata,
    boolean done,
    Status error,
    RESP response
) {
    /**
     * Error status information.
     */
    public record Status(
        int code,
        String message,
        List<Map<String, Object>> details
    ) {}
}

/**
 * States of batch jobs.
 */
public enum BatchJobState {
    BATCH_STATE_PENDING,      // Job is queued
    BATCH_STATE_RUNNING,      // Job is processing
    BATCH_STATE_SUCCEEDED,    // Job completed successfully
    BATCH_STATE_FAILED,       // Job failed
    BATCH_STATE_CANCELLED,    // Job was cancelled
    BATCH_STATE_EXPIRED,      // Job expired before completion
    UNSPECIFIED               // State unknown
}

JSON Lines Writing

Interface for writing batch request files in JSON Lines format.

/**
 * Writer for JSON Lines format files.
 * Each line is a separate JSON object.
 */
public interface JsonLinesWriter {
    /**
     * Writes a single object as a JSON line.
     * @param object Object to serialize and write
     * @throws IOException if an I/O error occurs while writing
     */
    void write(Object object) throws IOException;

    /**
     * Writes multiple objects as JSON lines.
     * @param objects Iterable of objects to write
     * @throws IOException if an I/O error occurs while writing
     */
    void write(Iterable<?> objects) throws IOException;

    /**
     * Flushes any buffered data to the underlying stream.
     * @throws IOException if an I/O error occurs while flushing
     */
    void flush() throws IOException;

    /**
     * Closes the writer and releases resources.
     * @throws IOException if an I/O error occurs while closing
     */
    void close() throws IOException;
}

/**
 * Factory for creating JsonLinesWriter instances.
 */
public class JsonLinesWriters {
    /**
     * Creates a streaming writer for a file path.
     * @param path Path to write JSON Lines file
     * @return JsonLinesWriter instance
     * @throws IOException if an I/O error occurs while creating the writer
     */
    public static JsonLinesWriter streaming(Path path) throws IOException;

    /**
     * Creates a streaming writer for an output stream.
     * @param outputStream Stream to write to
     * @return JsonLinesWriter instance
     */
    public static JsonLinesWriter streaming(OutputStream outputStream);
}

Batch Response Types

Sealed interface hierarchy representing different states and outcomes of batch operations.

/**
 * Sealed interface for batch operation responses.
 * Can be incomplete, successful, or error state.
 */
public sealed interface BatchResponse<T>
    permits BatchIncomplete, BatchSuccess, BatchError {
}

/**
 * Batch job is still processing (pending or running).
 */
public record BatchIncomplete<T>(
    /**
     * Identifier for the batch job.
     */
    BatchName batchName,

    /**
     * Current state of the batch job.
     */
    BatchJobState state
) implements BatchResponse<T> {}

/**
 * Batch job completed successfully.
 */
public record BatchSuccess<T>(
    /**
     * Identifier for the batch job.
     */
    BatchName batchName,

    /**
     * List of successful response items.
     */
    List<T> responses,

    /**
     * Optional list of errors for individual requests that failed.
     */
    List<Operation.Status> errors
) implements BatchResponse<T> {}

/**
 * Batch job failed with an error.
 */
public record BatchError<T>(
    /**
     * Identifier for the batch job.
     */
    BatchName batchName,

    /**
     * Error code.
     */
    int code,

    /**
     * Error message describing the failure.
     */
    String message,

    /**
     * Job state at time of error.
     */
    BatchJobState state,

    /**
     * Detailed error information.
     */
    List<Map<String, Object>> details
) implements BatchResponse<T> {}

/**
 * Paginated list of batch jobs.
 */
public record BatchList<T>(
    /**
     * Token for retrieving the next page of results.
     */
    String pageToken,

    /**
     * List of batch responses for this page.
     */
    List<BatchResponse<T>> responses
) {}

BatchName Record

Strongly-typed batch job identifier.

/**
 * Identifier for a batch job.
 * Must start with "batches/".
 */
public record BatchName(
    /**
     * Batch identifier value.
     * Format: "batches/{batch-id}"
     */
    String value
) {
    /**
     * Validates batch name format.
     * @throws IllegalArgumentException if format is invalid
     */
    public BatchName {
        if (value == null || !value.startsWith("batches/")) {
            throw new IllegalArgumentException(
                "Batch name must start with 'batches/'"
            );
        }
    }
}

BatchJobState Enum

Enum representing the lifecycle states of a batch job.

/**
 * States of a batch processing job.
 */
public enum BatchJobState {
    /**
     * Batch is queued and waiting to start.
     */
    BATCH_STATE_PENDING,

    /**
     * Batch is currently being processed.
     */
    BATCH_STATE_RUNNING,

    /**
     * Batch completed successfully.
     */
    BATCH_STATE_SUCCEEDED,

    /**
     * Batch failed with an error.
     */
    BATCH_STATE_FAILED,

    /**
     * Batch was cancelled by user.
     */
    BATCH_STATE_CANCELLED,

    /**
     * Batch expired before completion (24 hour limit).
     */
    BATCH_STATE_EXPIRED,

    /**
     * State is unspecified or unknown.
     */
    UNSPECIFIED
}

BatchFileRequest Record

Record wrapping a request with a unique key for batch file operations.

/**
 * Wrapper for batch file requests with unique identifiers.
 * Used when writing requests to batch files.
 */
public record BatchFileRequest<REQ>(
    /**
     * Unique identifier for this request within the batch.
     * Used to correlate requests with responses.
     */
    String key,

    /**
     * The actual request payload.
     */
    REQ request
) {}

Operation.Status Record

Error status information for failed batch operations.

/**
 * Status information for operations, particularly errors.
 */
public record Status(
    /**
     * Error code (0 = success, non-zero = error).
     */
    int code,

    /**
     * Human-readable error message.
     */
    String message,

    /**
     * Additional structured error details.
     */
    List<Map<String, Object>> details
) {}

Usage Examples

Basic Inline Batch - Chat

import dev.langchain4j.model.googleai.GoogleAiGeminiBatchChatModel;
import dev.langchain4j.data.message.UserMessage;
import dev.langchain4j.model.chat.request.ChatRequest;

GoogleAiGeminiBatchChatModel batchModel = GoogleAiGeminiBatchChatModel.builder()
    .apiKey(System.getenv("GOOGLE_AI_API_KEY"))
    .modelName("gemini-2.5-flash")
    .build();

// Create multiple chat requests
List<ChatRequest> requests = List.of(
    ChatRequest.builder()
        .messages(UserMessage.from("What is the capital of France?"))
        .build(),
    ChatRequest.builder()
        .messages(UserMessage.from("What is the capital of Germany?"))
        .build(),
    ChatRequest.builder()
        .messages(UserMessage.from("What is the capital of Spain?"))
        .build()
);

// Submit batch job
BatchResponse<ChatResponse> response = batchModel.createBatchInline(
    "Country Capitals Batch",
    100L, // priority
    requests
);

// Check response type
switch (response) {
    case BatchIncomplete<ChatResponse> incomplete ->
        System.out.println("Batch submitted: " + incomplete.batchName().value());
    case BatchSuccess<ChatResponse> success ->
        success.responses().forEach(r ->
            System.out.println(r.aiMessage().text())
        );
    case BatchError<ChatResponse> error ->
        System.err.println("Batch failed: " + error.message());
}

Batch from File - Chat

import dev.langchain4j.model.googleai.GeminiFiles;
import dev.langchain4j.model.googleai.jsonl.JsonLinesWriters;
import dev.langchain4j.model.googleai.BatchRequestResponse.BatchFileRequest;
import java.nio.file.Path;

// Step 1: Create batch file
Path batchFile = Path.of("chat-batch.jsonl");
try (JsonLinesWriter writer = JsonLinesWriters.streaming(batchFile)) {
    List<BatchFileRequest<ChatRequest>> fileRequests = List.of(
        new BatchFileRequest<>("req1", ChatRequest.builder()
            .messages(UserMessage.from("Summarize quantum computing"))
            .build()),
        new BatchFileRequest<>("req2", ChatRequest.builder()
            .messages(UserMessage.from("Explain neural networks"))
            .build())
    );

    batchModel.writeBatchToFile(writer, fileRequests);
}

// Step 2: Upload file
GeminiFiles filesClient = GeminiFiles.builder()
    .apiKey(System.getenv("GOOGLE_AI_API_KEY"))
    .build();

GeminiFile uploadedFile = filesClient.uploadFile(batchFile, "Chat Batch Requests");

// Step 3: Create batch from file
BatchResponse<ChatResponse> response = batchModel.createBatchFromFile(
    "AI Explanations Batch",
    uploadedFile
);

if (response instanceof BatchIncomplete<ChatResponse> incomplete) {
    BatchName batchName = incomplete.batchName();
    System.out.println("Batch submitted: " + batchName.value());

    // Step 4: Poll for results (after some time)
    Thread.sleep(3600000); // Wait 1 hour

    BatchResponse<ChatResponse> results = batchModel.retrieveBatchResults(batchName);
    if (results instanceof BatchSuccess<ChatResponse> success) {
        System.out.println("Batch completed with " + success.responses().size() + " responses");
        success.responses().forEach(r -> System.out.println(r.aiMessage().text()));
    }
}

Batch Embeddings

import dev.langchain4j.model.googleai.GoogleAiGeminiBatchEmbeddingModel;
import dev.langchain4j.data.segment.TextSegment;

GoogleAiGeminiBatchEmbeddingModel batchEmbedding = GoogleAiGeminiBatchEmbeddingModel.builder()
    .apiKey(System.getenv("GOOGLE_AI_API_KEY"))
    .modelName("text-embedding-004")
    .taskType(GoogleAiEmbeddingModel.TaskType.SEMANTIC_SIMILARITY)
    .build();

List<TextSegment> segments = List.of(
    TextSegment.from("The quick brown fox"),
    TextSegment.from("Jumps over the lazy dog"),
    TextSegment.from("Machine learning is fascinating")
);

BatchResponse<Embedding> response = batchEmbedding.createBatchInline(
    "Text Embeddings Batch",
    null, // no priority
    segments
);

if (response instanceof BatchIncomplete<Embedding> incomplete) {
    System.out.println("Embedding batch started: " + incomplete.batchName().value());
}

Batch Image Generation

import dev.langchain4j.model.googleai.GoogleAiGeminiBatchImageModel;
import dev.langchain4j.model.googleai.GoogleAiGeminiBatchImageModel.ImageGenerationRequest;

GoogleAiGeminiBatchImageModel batchImage = GoogleAiGeminiBatchImageModel.builder()
    .apiKey(System.getenv("GOOGLE_AI_API_KEY"))
    .modelName("gemini-2.5-flash")
    .build();

List<ImageGenerationRequest> requests = List.of(
    new ImageGenerationRequest("A serene mountain landscape at sunset"),
    new ImageGenerationRequest("A futuristic city with flying cars"),
    new ImageGenerationRequest("An abstract painting of emotions")
);

BatchResponse<Response<Image>> response = batchImage.createBatchInline(
    "Art Generation Batch",
    50L,
    requests
);

if (response instanceof BatchIncomplete<Response<Image>> incomplete) {
    System.out.println("Image batch started: " + incomplete.batchName().value());
}

Listing Batch Jobs

// List all batch jobs with pagination
BatchList<ChatResponse> batchList = batchModel.listBatchJobs(10, null);

System.out.println("Batch jobs:");
for (BatchResponse<ChatResponse> batch : batchList.responses()) {
    switch (batch) {
        case BatchIncomplete<ChatResponse> inc ->
            System.out.println("- " + inc.batchName().value() + ": " + inc.state());
        case BatchSuccess<ChatResponse> suc ->
            System.out.println("- " + suc.batchName().value() + ": SUCCESS (" +
                suc.responses().size() + " responses)");
        case BatchError<ChatResponse> err ->
            System.out.println("- " + err.batchName().value() + ": ERROR - " + err.message());
    }
}

// Get next page if available
if (batchList.pageToken() != null) {
    BatchList<ChatResponse> nextPage = batchModel.listBatchJobs(10, batchList.pageToken());
    // Process next page...
}

Cancelling a Batch Job

BatchName batchName = new BatchName("projects/123/locations/us-central1/batchPredictionJobs/456");

try {
    batchModel.cancelBatchJob(batchName);
    System.out.println("Batch job cancelled");
} catch (Exception e) {
    System.err.println("Failed to cancel: " + e.getMessage());
}

Deleting a Batch Job

BatchName batchName = new BatchName("projects/123/locations/us-central1/batchPredictionJobs/456");

try {
    batchModel.deleteBatchJob(batchName);
    System.out.println("Batch job deleted");
} catch (Exception e) {
    System.err.println("Failed to delete: " + e.getMessage());
}

Pattern Matching for Batch Results

BatchResponse<ChatResponse> response = batchModel.retrieveBatchResults(batchName);

String message = switch (response) {
    case BatchIncomplete<ChatResponse>(var name, var state) ->
        "Batch " + name.value() + " is " + state;

    case BatchSuccess<ChatResponse>(var name, var responses, var errors) -> {
        StringBuilder sb = new StringBuilder("Batch " + name.value() + " completed:\n");
        sb.append("- Successes: ").append(responses.size()).append("\n");
        if (errors != null) {
            sb.append("- Errors: ").append(errors.size());
        }
        yield sb.toString();
    }

    case BatchError<ChatResponse>(var name, var code, var msg, var state, var details) ->
        "Batch " + name.value() + " failed (code " + code + "): " + msg;
};

System.out.println(message);

Cost Savings

Batch processing offers significant cost advantages:

50% cost reduction compared to synchronous API calls
24-hour SLO for processing completion
Ideal for non-time-sensitive workloads (data analysis, bulk content generation, large-scale embeddings)

Best Practices

File-based batches: For large jobs (>100 requests), use file-based batching to avoid request size limits
Polling: Implement exponential backoff when polling for results
Error handling: Check for both batch-level errors and individual request errors in BatchSuccess
Cleanup: Delete completed batch jobs to free up resources
Priority: Use priority parameter for urgent batches (higher = higher priority)
Request keys: Use meaningful keys in BatchFileRequest for easier result mapping

Limitations

Maximum batch size varies by model and request type
24-hour processing window before expiration
Results are available for 48 hours after completion
Cannot modify batch after submission
Batch operations are experimental and subject to change

Install with Tessl CLI

npx tessl i tessl/maven-dev-langchain4j--langchain4j-google-ai-gemini@1.11.0

docs

advanced-configuration.md

tessl/maven-dev-langchain4j--langchain4j-google-ai-gemini