or run

npx @tessl/cli init
Log in

Version

Tile

Overview

Evals

Files

docs

configuration-utilities.mdevaluation-metrics.mdindex.mdrequest-response-api.md
tile.json

tessl/maven-org-elasticsearch-plugin--rank-eval-client

Elasticsearch plugin providing comprehensive ranking evaluation capabilities for search quality assessment.

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
mavenpkg:maven/org.elasticsearch.plugin/rank-eval-client@7.17.x

To install, run

npx @tessl/cli install tessl/maven-org-elasticsearch-plugin--rank-eval-client@7.17.0

index.mddocs/

Elasticsearch Rank Eval Plugin

The Elasticsearch Rank Eval plugin provides comprehensive ranking evaluation capabilities for search quality assessment. It implements multiple information retrieval metrics including Precision@K, Recall@K, Mean Reciprocal Rank (MRR), Discounted Cumulative Gain (DCG), and Expected Reciprocal Rank (ERR) to evaluate the quality of ranked search results against manually rated document sets.

Package Information

  • Package Name: org.elasticsearch.plugin/rank-eval-client
  • Package Type: maven
  • Language: Java
  • Minimum Java Version: 11
  • Elasticsearch Version: 7.17.29
  • Installation: 
    <dependency>
    <groupId>org.elasticsearch.plugin</groupId>
    <artifactId>rank-eval-client</artifactId>
    <version>7.17.29</version>
</dependency>

Core Imports

// Main request/response classes
import org.elasticsearch.index.rankeval.RankEvalRequest;
import org.elasticsearch.index.rankeval.RankEvalResponse;
import org.elasticsearch.index.rankeval.RankEvalSpec;
import org.elasticsearch.index.rankeval.RankEvalAction;

// Evaluation metrics
import org.elasticsearch.index.rankeval.PrecisionAtK;
import org.elasticsearch.index.rankeval.RecallAtK;
import org.elasticsearch.index.rankeval.MeanReciprocalRank;
import org.elasticsearch.index.rankeval.DiscountedCumulativeGain;
import org.elasticsearch.index.rankeval.ExpectedReciprocalRank;

// Supporting classes
import org.elasticsearch.index.rankeval.RatedRequest;
import org.elasticsearch.index.rankeval.RatedDocument;
import org.elasticsearch.index.rankeval.EvalQueryQuality;

Basic Usage

// Create evaluation metric
PrecisionAtK metric = new PrecisionAtK(1, false, 10);

// Create rated documents for a query
List<RatedDocument> ratedDocs = Arrays.asList(
    new RatedDocument("test_index", "doc1", 2),
    new RatedDocument("test_index", "doc2", 1),
    new RatedDocument("test_index", "doc3", 0)
);

// Create search request
SearchSourceBuilder searchSource = SearchSourceBuilder.searchSource()
    .query(QueryBuilders.matchQuery("title", "elasticsearch"));

// Create rated request
RatedRequest ratedRequest = new RatedRequest(
    "test_query", 
    Arrays.asList("test_index"), 
    searchSource, 
    ratedDocs
);

// Create evaluation specification
RankEvalSpec spec = new RankEvalSpec(
    Arrays.asList(ratedRequest), 
    metric
);

// Create and execute rank evaluation request
RankEvalRequest request = new RankEvalRequest(spec, new String[]{"test_index"});
RankEvalResponse response = client.execute(RankEvalAction.INSTANCE, request).get();

// Access results
double overallScore = response.getMetricScore();
Map<String, EvalQueryQuality> queryResults = response.getPartialResults();

Capabilities

Request and Response API

The core API for creating and executing ranking evaluation requests, including request builders, response handling, and evaluation specifications.

// Main request class
public class RankEvalRequest extends ActionRequest implements IndicesRequest.Replaceable {
    public RankEvalRequest(RankEvalSpec rankingEvaluationSpec, String[] indices);
    public RankEvalSpec getRankEvalSpec();
    public void setRankEvalSpec(RankEvalSpec task);
}

// Main response class  
public class RankEvalResponse extends ActionResponse implements ToXContentObject {
    public double getMetricScore();
    public Map<String, EvalQueryQuality> getPartialResults();
    public Map<String, Exception> getFailures();
}

// Evaluation specification
public class RankEvalSpec implements Writeable, ToXContentObject {
    public RankEvalSpec(List<RatedRequest> ratedRequests, EvaluationMetric metric);
    public EvaluationMetric getMetric();
    public List<RatedRequest> getRatedRequests();
}

Request and Response API Documentation

Evaluation Metrics

Implementation of information retrieval metrics for ranking quality assessment, including Precision@K, Recall@K, MRR, DCG, and ERR with configurable parameters.

// Base interface for all metrics
public interface EvaluationMetric extends ToXContentObject, NamedWriteable {
    EvalQueryQuality evaluate(String taskId, SearchHit[] hits, List<RatedDocument> ratedDocs);
    default double combine(Collection<EvalQueryQuality> partialResults);
}

// Precision@K implementation
public class PrecisionAtK implements EvaluationMetric {
    public static final String NAME = "precision";
    public PrecisionAtK(int relevantRatingThreshold, boolean ignoreUnlabeled, int k);
}

// Recall@K implementation  
public class RecallAtK implements EvaluationMetric {
    public static final String NAME = "recall";
    public RecallAtK(int relevantRatingThreshold, int k);
}

Evaluation Metrics Documentation

Configuration and Utilities

Supporting classes for document rating, search hit handling, query quality evaluation, and configuration management.

// Document with rating
public class RatedDocument implements Writeable, ToXContentObject {
    public RatedDocument(String index, String id, int rating);
    public DocumentKey getKey();
    public int getRating();
}

// Query with rated documents
public class RatedRequest implements Writeable, ToXContentObject {
    public RatedRequest(String id, List<String> indices, SearchSourceBuilder request, List<RatedDocument> ratedDocs);
    public String getId();
    public List<RatedDocument> getRatedDocs();
}

// Query evaluation result
public class EvalQueryQuality implements ToXContentFragment, Writeable {
    public String getId();
    public double metricScore();
    public MetricDetail getMetricDetails();
}

Configuration and Utilities Documentation

Architecture

The Rank Eval plugin follows Elasticsearch's plugin architecture with these key components:

  • Plugin Registration: RankEvalPlugin registers actions and REST handlers
  • Transport Layer: TransportRankEvalAction handles distributed execution
  • REST API: RestRankEvalAction provides HTTP endpoints at /_rank_eval
  • Metric System: Pluggable evaluation metrics implementing EvaluationMetric interface
  • Request Processing: Evaluation requests processed across cluster nodes with result aggregation

The plugin supports both individual query evaluation and batch evaluation with template-based queries for systematic ranking quality assessment.