CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl/maven-org-elasticsearch-plugin--rank-eval-client

Elasticsearch plugin providing comprehensive ranking evaluation capabilities for search quality assessment.

—

Pending
Quality

Pending

Does it follow best practices?

Impact

Pending

No eval scenarios have been run

SecuritybySnyk

Pending

The risk profile of this skill

Overview
Eval results
Files

Elasticsearch Rank Eval Plugin

The Elasticsearch Rank Eval plugin provides comprehensive ranking evaluation capabilities for search quality assessment. It implements multiple information retrieval metrics including Precision@K, Recall@K, Mean Reciprocal Rank (MRR), Discounted Cumulative Gain (DCG), and Expected Reciprocal Rank (ERR) to evaluate the quality of ranked search results against manually rated document sets.

Package Information

  • Package Name: org.elasticsearch.plugin/rank-eval-client
  • Package Type: maven
  • Language: Java
  • Minimum Java Version: 11
  • Elasticsearch Version: 7.17.29
  • Installation: 
    <dependency>
    <groupId>org.elasticsearch.plugin</groupId>
    <artifactId>rank-eval-client</artifactId>
    <version>7.17.29</version>
</dependency>

Core Imports

// Main request/response classes
import org.elasticsearch.index.rankeval.RankEvalRequest;
import org.elasticsearch.index.rankeval.RankEvalResponse;
import org.elasticsearch.index.rankeval.RankEvalSpec;
import org.elasticsearch.index.rankeval.RankEvalAction;

// Evaluation metrics
import org.elasticsearch.index.rankeval.PrecisionAtK;
import org.elasticsearch.index.rankeval.RecallAtK;
import org.elasticsearch.index.rankeval.MeanReciprocalRank;
import org.elasticsearch.index.rankeval.DiscountedCumulativeGain;
import org.elasticsearch.index.rankeval.ExpectedReciprocalRank;

// Supporting classes
import org.elasticsearch.index.rankeval.RatedRequest;
import org.elasticsearch.index.rankeval.RatedDocument;
import org.elasticsearch.index.rankeval.EvalQueryQuality;

Basic Usage

// Create evaluation metric
PrecisionAtK metric = new PrecisionAtK(1, false, 10);

// Create rated documents for a query
List<RatedDocument> ratedDocs = Arrays.asList(
    new RatedDocument("test_index", "doc1", 2),
    new RatedDocument("test_index", "doc2", 1),
    new RatedDocument("test_index", "doc3", 0)
);

// Create search request
SearchSourceBuilder searchSource = SearchSourceBuilder.searchSource()
    .query(QueryBuilders.matchQuery("title", "elasticsearch"));

// Create rated request
RatedRequest ratedRequest = new RatedRequest(
    "test_query", 
    Arrays.asList("test_index"), 
    searchSource, 
    ratedDocs
);

// Create evaluation specification
RankEvalSpec spec = new RankEvalSpec(
    Arrays.asList(ratedRequest), 
    metric
);

// Create and execute rank evaluation request
RankEvalRequest request = new RankEvalRequest(spec, new String[]{"test_index"});
RankEvalResponse response = client.execute(RankEvalAction.INSTANCE, request).get();

// Access results
double overallScore = response.getMetricScore();
Map<String, EvalQueryQuality> queryResults = response.getPartialResults();

Capabilities

Request and Response API

The core API for creating and executing ranking evaluation requests, including request builders, response handling, and evaluation specifications.

// Main request class
public class RankEvalRequest extends ActionRequest implements IndicesRequest.Replaceable {
    public RankEvalRequest(RankEvalSpec rankingEvaluationSpec, String[] indices);
    public RankEvalSpec getRankEvalSpec();
    public void setRankEvalSpec(RankEvalSpec task);
}

// Main response class  
public class RankEvalResponse extends ActionResponse implements ToXContentObject {
    public double getMetricScore();
    public Map<String, EvalQueryQuality> getPartialResults();
    public Map<String, Exception> getFailures();
}

// Evaluation specification
public class RankEvalSpec implements Writeable, ToXContentObject {
    public RankEvalSpec(List<RatedRequest> ratedRequests, EvaluationMetric metric);
    public EvaluationMetric getMetric();
    public List<RatedRequest> getRatedRequests();
}

Request and Response API Documentation

Evaluation Metrics

Implementation of information retrieval metrics for ranking quality assessment, including Precision@K, Recall@K, MRR, DCG, and ERR with configurable parameters.

// Base interface for all metrics
public interface EvaluationMetric extends ToXContentObject, NamedWriteable {
    EvalQueryQuality evaluate(String taskId, SearchHit[] hits, List<RatedDocument> ratedDocs);
    default double combine(Collection<EvalQueryQuality> partialResults);
}

// Precision@K implementation
public class PrecisionAtK implements EvaluationMetric {
    public static final String NAME = "precision";
    public PrecisionAtK(int relevantRatingThreshold, boolean ignoreUnlabeled, int k);
}

// Recall@K implementation  
public class RecallAtK implements EvaluationMetric {
    public static final String NAME = "recall";
    public RecallAtK(int relevantRatingThreshold, int k);
}

Evaluation Metrics Documentation

Configuration and Utilities

Supporting classes for document rating, search hit handling, query quality evaluation, and configuration management.

// Document with rating
public class RatedDocument implements Writeable, ToXContentObject {
    public RatedDocument(String index, String id, int rating);
    public DocumentKey getKey();
    public int getRating();
}

// Query with rated documents
public class RatedRequest implements Writeable, ToXContentObject {
    public RatedRequest(String id, List<String> indices, SearchSourceBuilder request, List<RatedDocument> ratedDocs);
    public String getId();
    public List<RatedDocument> getRatedDocs();
}

// Query evaluation result
public class EvalQueryQuality implements ToXContentFragment, Writeable {
    public String getId();
    public double metricScore();
    public MetricDetail getMetricDetails();
}

Configuration and Utilities Documentation

Architecture

The Rank Eval plugin follows Elasticsearch's plugin architecture with these key components:

  • Plugin Registration: RankEvalPlugin registers actions and REST handlers
  • Transport Layer: TransportRankEvalAction handles distributed execution
  • REST API: RestRankEvalAction provides HTTP endpoints at /_rank_eval
  • Metric System: Pluggable evaluation metrics implementing EvaluationMetric interface
  • Request Processing: Evaluation requests processed across cluster nodes with result aggregation

The plugin supports both individual query evaluation and batch evaluation with template-based queries for systematic ranking quality assessment.

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
mavenpkg:maven/org.elasticsearch.plugin/rank-eval-client@7.17.x
Publish Source
CLI
Badge
tessl/maven-org-elasticsearch-plugin--rank-eval-client badge