tessl/maven-org-elasticsearch-plugin--rank-eval-client

Elasticsearch plugin providing comprehensive ranking evaluation capabilities for search quality assessment.

—

Pending

Overview

Eval results

Files

Configuration and Utilities

Name: tessl/maven-org-elasticsearch-plugin--rank-eval-client
Author: tessl

This section covers supporting classes for document rating, search hit handling, query quality evaluation, configuration management, and plugin infrastructure.

Document and Rating Classes

RatedDocument

Represents a document with its relevance rating for a specific query.

public class RatedDocument implements Writeable, ToXContentObject {
    // Constructor
    public RatedDocument(String index, String id, int rating);
    
    // Property access
    public DocumentKey getKey();
    public int getRating();
    
    // Serialization methods
    public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException;
    public static RatedDocument fromXContent(XContentParser parser) throws IOException;
    public void writeTo(StreamOutput out) throws IOException;
    
    // Equality and comparison
    public boolean equals(Object obj);
    public int hashCode();
    
    // Nested DocumentKey class
    public static class DocumentKey {
        public DocumentKey(String index, String id);
        public String getIndex();
        public String getId();
        public boolean equals(Object obj);
        public int hashCode();
    }
}

Usage:

// Create rated documents with different relevance levels
List<RatedDocument> ratedDocs = Arrays.asList(
    new RatedDocument("products", "laptop_pro", 3),    // highly relevant
    new RatedDocument("products", "laptop_basic", 2),  // relevant
    new RatedDocument("products", "tablet", 1),        // somewhat relevant
    new RatedDocument("products", "phone", 0)          // not relevant
);

// Access document properties
for (RatedDocument doc : ratedDocs) {
    DocumentKey key = doc.getKey();
    System.out.println("Document " + key.getId() + " in index " + key.getIndex() + 
                      " has rating: " + doc.getRating());
}

RatedSearchHit

Wrapper combining a search hit with its optional rating.

public class RatedSearchHit implements Writeable, ToXContentObject {
    // Constructor
    public RatedSearchHit(SearchHit hit, OptionalInt rating);
    
    // Property access
    public SearchHit getSearchHit();
    public OptionalInt getRating();
    
    // Serialization methods
    public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException;
    public void writeTo(StreamOutput out) throws IOException;
    
    // Equality and comparison
    public boolean equals(Object obj);
    public int hashCode();
}

Usage:

// Join search results with ratings
SearchHit[] searchHits = searchResponse.getHits().getHits();
List<RatedDocument> ratedDocs = getRatedDocuments();

List<RatedSearchHit> ratedSearchHits = EvaluationMetric.joinHitsWithRatings(searchHits, ratedDocs);

// Process rated search hits
for (RatedSearchHit ratedHit : ratedSearchHits) {
    SearchHit hit = ratedHit.getSearchHit();
    OptionalInt rating = ratedHit.getRating();
    
    if (rating.isPresent()) {
        System.out.println("Document " + hit.getId() + " has rating: " + rating.getAsInt());
    } else {
        System.out.println("Document " + hit.getId() + " is unrated");
    }
}

Query Evaluation Results

EvalQueryQuality

Contains the evaluation result for a single query, including metric score and detailed breakdown.

public class EvalQueryQuality implements ToXContentFragment, Writeable {
    // Constructors
    public EvalQueryQuality(String id, double metricScore);
    public EvalQueryQuality(String id, double metricScore, MetricDetail optionalMetricDetails, List<RatedSearchHit> ratedHits);
    
    // Property access
    public String getId();
    public double metricScore();
    public MetricDetail getMetricDetails();
    public List<RatedSearchHit> getHitsAndRatings();
    
    // Mutation methods
    public void setMetricDetails(MetricDetail breakdown);
    public void addHitsAndRatings(List<RatedSearchHit> ratedSearchHits);
    
    // Serialization methods
    public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException;
    public static EvalQueryQuality fromXContent(XContentParser parser, String queryId) throws IOException;
    public void writeTo(StreamOutput out) throws IOException;
    
    // Comparison and equality
    public boolean equals(Object obj);
    public int hashCode();
}

Usage:

// Access query evaluation results
Map<String, EvalQueryQuality> queryResults = response.getPartialResults();

for (Map.Entry<String, EvalQueryQuality> entry : queryResults.entrySet()) {
    String queryId = entry.getKey();
    EvalQueryQuality quality = entry.getValue();
    
    // Get basic metrics
    double score = quality.metricScore();
    System.out.println("Query " + queryId + " score: " + String.format("%.4f", score));
    
    // Access detailed breakdown if available
    MetricDetail details = quality.getMetricDetails();
    if (details != null) {
        // Cast to specific detail type based on metric used
        System.out.println("Detailed breakdown available");
    }
    
    // Access individual search hits with ratings
    List<RatedSearchHit> hitsAndRatings = quality.getHitsAndRatings();
    System.out.println("Evaluated " + hitsAndRatings.size() + " search hits");
}

MetricDetail

Base class for metric-specific detailed breakdown information.

public abstract class MetricDetail implements NamedWriteable, ToXContentFragment {
    // Abstract methods implemented by metric-specific detail classes
    public abstract XContentBuilder innerToXContent(XContentBuilder builder, Params params) throws IOException;
    
    // Common serialization interface
    public XContentBuilder toXContent(XContentBuilder builder, Params params) throws IOException;
}

Metric-Specific Detail Classes:

// PrecisionAtK detailed results
public static class Detail extends MetricDetail {
    public int getRelevantRetrieved();
    public int getRetrieved();
}

// RecallAtK detailed results  
public static class Detail extends MetricDetail {
    public long getRelevantRetrieved();
    public long getRelevant();
}

// MeanReciprocalRank detailed results
public static class Detail extends MetricDetail {
    public int getFirstRelevantRank();
}

// DiscountedCumulativeGain detailed results
public static class Detail extends MetricDetail {
    public double getDcg();
    public double getIdealDcg();
    public double getNdcg();
}

// ExpectedReciprocalRank detailed results
public static class Detail extends MetricDetail {
    public double getErr();
}

Plugin Infrastructure

RankEvalPlugin

Main plugin class that registers the rank evaluation functionality with Elasticsearch.

public class RankEvalPlugin extends Plugin implements ActionPlugin {
    // Plugin lifecycle methods
    public List<ActionHandler<? extends ActionRequest, ? extends ActionResponse>> getActions();
    public List<RestHandler> getRestHandlers(Settings settings, RestController restController, 
                                            ClusterSettings clusterSettings, IndexScopedSettings indexScopedSettings,
                                            SettingsFilter settingsFilter, IndexNameExpressionResolver indexNameExpressionResolver,
                                            Supplier<DiscoveryNodes> nodesInCluster);
    
    // Serialization registry
    public List<NamedWriteableRegistry.Entry> getNamedWriteables();
    public List<Entry> getNamedXContent();
}

RankEvalAction

Action type definition for rank evaluation operations.

public class RankEvalAction extends ActionType<RankEvalResponse> {
    // Singleton instance
    public static final RankEvalAction INSTANCE = new RankEvalAction();
    
    // Action name constant
    public static final String NAME = "indices:data/read/rank_eval";
    
    // Private constructor - use INSTANCE
    private RankEvalAction();
}

RestRankEvalAction

REST endpoint handler providing HTTP access to rank evaluation functionality.

public class RestRankEvalAction extends BaseRestHandler {
    // Supported HTTP methods and paths
    public List<Route> routes();
    public String getName();
    
    // Request processing
    public RestChannelConsumer prepareRequest(RestRequest request, NodeClient client) throws IOException;
    
    // Parameter parsing
    protected static IndicesOptions parseIndicesOptions(RestRequest restRequest);
    protected static SearchType parseSearchType(RestRequest restRequest);
}

Supported Endpoints:

GET /_rank_eval
POST /_rank_eval
GET /{index}/_rank_eval
POST /{index}/_rank_eval

TransportRankEvalAction

Transport layer implementation handling distributed execution across cluster nodes.

public class TransportRankEvalAction extends HandledTransportAction<RankEvalRequest, RankEvalResponse> {
    // Constructor
    public TransportRankEvalAction(TransportService transportService, ClusterService clusterService, 
                                  ThreadPool threadPool, ActionFilters actionFilters, 
                                  IndexNameExpressionResolver indexNameExpressionResolver, Client client);
    
    // Core execution method
    protected void doExecute(Task task, RankEvalRequest request, ActionListener<RankEvalResponse> listener);
}

Content Providers and Parsers

RankEvalNamedXContentProvider

Provides named XContent parsers for rank evaluation components, enabling JSON/XContent parsing.

public class RankEvalNamedXContentProvider {
    // Get list of named XContent parsers
    public List<NamedXContentRegistry.Entry> getNamedXContentParsers();
}

Configuration Examples

Basic Configuration

// Simple evaluation with default settings
PrecisionAtK metric = new PrecisionAtK();
RankEvalSpec spec = new RankEvalSpec(ratedRequests, metric);
RankEvalRequest request = new RankEvalRequest(spec, new String[]{"products"});

Advanced Configuration

// Custom metric configuration
DiscountedCumulativeGain ndcg = new DiscountedCumulativeGain(true, null, 20);

// Specification with templates and concurrency control
RankEvalSpec spec = new RankEvalSpec(ratedRequests, ndcg, templates);
spec.setMaxConcurrentSearches(5);

// Request with custom indices options
RankEvalRequest request = new RankEvalRequest(spec, new String[]{"index1", "index2"});
request.indicesOptions(IndicesOptions.strictExpandOpen());
request.searchType(SearchType.DFS_QUERY_THEN_FETCH);

Template Configuration

// Create query template
Script template = new Script(
    ScriptType.INLINE,
    "mustache",
    """
    {
      "query": {
        "bool": {
          "must": [
            {"match": {"{{title_field}}": "{{search_term}}"}},
            {"range": {"price": {"lte": {{max_price}}}}}
          ]
        }
      },
      "sort": [{"_score": "desc"}, {"price": "asc"}]
    }
    """,
    Collections.emptyMap()
);

// Wrap in ScriptWithId
ScriptWithId scriptWithId = new ScriptWithId("product_search", template);

// Create rated request using template
RatedRequest templateRequest = new RatedRequest(
    "search_laptops",
    "product_search",
    Map.of(
        "title_field", "title",
        "search_term", "laptop",
        "max_price", 2000
    ),
    ratedDocs
);

// Include template in specification
RankEvalSpec spec = new RankEvalSpec(
    Arrays.asList(templateRequest),
    new PrecisionAtK(10, 2, false),
    Arrays.asList(scriptWithId)
);

Error Handling and Validation

// Comprehensive error handling
try {
    // Validate request before execution
    ActionRequestValidationException validationException = request.validate();
    if (validationException != null) {
        System.err.println("Request validation failed: " + validationException.getMessage());
        return;
    }
    
    // Execute request
    RankEvalResponse response = client.execute(RankEvalAction.INSTANCE, request).get();
    
    // Check for partial failures
    Map<String, Exception> failures = response.getFailures();
    if (!failures.isEmpty()) {
        System.out.println("Some queries failed:");
        failures.forEach((queryId, exception) -> 
            System.err.println("  " + queryId + ": " + exception.getMessage())
        );
    }
    
    // Process successful results
    double overallScore = response.getMetricScore();
    Map<String, EvalQueryQuality> partialResults = response.getPartialResults();
    
    System.out.println("Overall metric score: " + String.format("%.4f", overallScore));
    System.out.println("Successful queries: " + partialResults.size());
    
} catch (ExecutionException e) {
    if (e.getCause() instanceof ElasticsearchException) {
        ElasticsearchException esException = (ElasticsearchException) e.getCause();
        System.err.println("Elasticsearch error: " + esException.getMessage());
        
        // Handle specific error types
        if (esException instanceof IndexNotFoundException) {
            System.err.println("One or more indices not found");
        } else if (esException instanceof SearchPhaseExecutionException) {
            System.err.println("Search phase execution failed");
        }
    }
} catch (InterruptedException e) {
    Thread.currentThread().interrupt();
    System.err.println("Request was interrupted");
}

Performance Optimization

// Optimize for large-scale evaluation
RankEvalSpec spec = new RankEvalSpec(ratedRequests, metric);

// Control concurrency to prevent resource exhaustion
spec.setMaxConcurrentSearches(3);

// Use appropriate search type for performance
request.searchType(SearchType.QUERY_THEN_FETCH);

// Configure indices options for efficiency
request.indicesOptions(IndicesOptions.lenientExpandOpen());

// For metrics that don't need full result sets, they may specify forced search size
OptionalInt forcedSize = metric.forcedSearchSize();
if (forcedSize.isPresent()) {
    System.out.println("Metric requires search size: " + forcedSize.getAsInt());
}

Install with Tessl CLI

npx tessl i tessl/maven-org-elasticsearch-plugin--rank-eval-client

docs

configuration-utilities.md

evaluation-metrics.md

index.md

request-response-api.md

tile.json

tessl/maven-org-elasticsearch-plugin--rank-eval-client

configuration-utilities.md.css-3qkkll{font-size:var(--chakra-font-sizes-sm);font-weight:var(--chakra-font-weights-normal);color:var(--chakra-colors-gray-300);}docs/

Configuration and Utilities

Document and Rating Classes

RatedDocument

RatedSearchHit

Query Evaluation Results

EvalQueryQuality

MetricDetail

Plugin Infrastructure

RankEvalPlugin

RankEvalAction

RestRankEvalAction

TransportRankEvalAction

Content Providers and Parsers

RankEvalNamedXContentProvider

Configuration Examples

Basic Configuration

Advanced Configuration

Template Configuration

Error Handling and Validation

Performance Optimization

configuration-utilities.mddocs/