testland/vector-search-recall-tests

Vector search benchmarking - recall@k vs latency tradeoffs, ground-truth construction via brute-force, HNSW tuning (M / ef_construct / ef per Qdrant docs), embedding-model-upgrade drift detection. Use ANN-Benchmarks framework for cross-engine comparison; per-engine clients (Qdrant, Weaviate, pgvector, Pinecone, Elasticsearch k-NN, Milvus) for in-product tests. Use when HNSW / IVF parameters are being tuned or an embedding model is swapped, and recall@k on the existing corpus has never been measured against brute-force ground truth.

Quality

90%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Per-engine clients and deprecated APIs

Name: testland/vector-search-recall-tests
Rating: 72 (1 reviews)
Author: testland

Engine-specific client code for vector-search-recall-tests. The core runnable sweep uses the Qdrant client inline in SKILL.md Step 3; other engines expose ef and search differently.

Weaviate v4

In Weaviate v4 ef is not a query argument: it is a collection vectorIndexConfig setting, so the sweep updates the collection config between rounds, then re-queries (per the Weaviate Python client docs):

from weaviate.classes.config import Reconfigure
from weaviate.classes.query import MetadataQuery

def set_weaviate_ef(client, ef, vector_name="default"):
    client.collections.use("Docs").config.update(
        vector_config=Reconfigure.Vectors.update(
            name=vector_name,
            vector_index_config=Reconfigure.VectorIndex.hnsw(ef=ef),
        ),
    )

def weaviate_search(client, query_vec, k=10):
    resp = client.collections.use("Docs").query.near_vector(
        near_vector=query_vec, limit=k, return_metadata=MetadataQuery(distance=True)
    )
    return [o.uuid for o in resp.objects]  # match against UUID-keyed ground truth

Deprecated / old APIs

qdrant-client < 1.18: client.search() was removed; use client.query_points() with SearchParams(hnsw_ef=ef) (SKILL.md Step 3).
Weaviate v3: query and ef APIs differ from v4; v4 moved ef to the collection vectorIndexConfig (see above).