wigolo find_similar

Hybrid semantic discovery: semantic embeddings + keyword search + web search, fused via Reciprocal Rank Fusion (RRF).

Quick Reference

// Find pages similar to a URL
{ "url": "https://docs.astro.build/en/getting-started/" }

// Find pages related to a concept
{ "concept": "JavaScript framework server-side rendering" }

// Scoped to specific domains
{ "url": "https://react.dev/reference/react/use", "include_domains": ["vuejs.org", "svelte.dev"] }

// Cache-only (no web fallback)
{ "url": "https://example.com/page", "include_web": false }

Parameters

Parameter	Type	Default	When to use
`url`	string	—	Find pages similar to this URL's content
`concept`	string	—	Find pages related to a text concept (no URL needed)
`max_results`	number	10	Cap at 50
`include_domains`	string[]	none	Scope results to specific sites
`exclude_domains`	string[]	none	Filter out domains
`include_cache`	boolean	true	Search local cache (fast, free)
`include_web`	boolean	true	Web fallback when cache is sparse
`mode`	string	"auto"	"auto", "cache", "web-expansion", "crawl-rank"
`threshold`	number	0	Hard post-filter on the raw fused score; 0 = no filtering. Filters `match_signals.fused_score`, not the normalized `relevance_score`
`include_ranking_debug`	boolean	false	Attach per-result `ranking_debug` with the raw ranks
`max_tokens_out`	number	none	Token-budget cap (cl100k-base)
`include_full_markdown`	boolean	false	Restore full body alongside evidence
`citation_format`	string	"numbered"	"numbered" / "json" / "anthropic_tags"

Provide either url or concept (not both).

How It Works

Embeds the input (URL content or concept text) into a vector.
Searches local cache via embedding similarity + keyword matching.
Falls back to web search if local hits are sparse.
Fuses all signals via 3-way Reciprocal Rank Fusion (RRF).
Returns ranked results. Each carries match_signals with the fused_score. Set include_ranking_debug: true to also attach a per-result ranking_debug object exposing the individual source ranks (fts5_rank, embedding_rank, web_rank, rrf_score) so you can audit disagreement between the three ranking sources.

Modes

auto (default) — pick the strategy from available signals.
cache — local hybrid only (keyword + semantic over the cache).
web-expansion — derive key terms and expand via web search.
crawl-rank — 1-hop crawl from the seed URL, embed, and cosine-rank the neighbours.

Cold-Start Signal

When the fused score from local signals is below threshold (env WIGOLO_FIND_SIMILAR_COLD_START_THRESHOLD), the response includes a cold_start string explaining why results came from web search. Pass it verbatim to the user.

Important: Build the Cache First

find_similar works best with a warm cache. Recommended workflow:

// Step 1: crawl to populate cache with embeddings
{ "url": "https://docs.framework.dev", "strategy": "sitemap", "max_pages": 20 }

// Step 2: now find_similar has real semantic signal
{ "url": "https://docs.framework.dev/getting-started" }

Anti-Patterns

DON'T use find_similar on a fresh install expecting embedding results — crawl first.
DON'T provide both url and concept — pick one.
DON'T use when you want web-only results — use search instead.

When NOT to use wigolo-find-similar

No local cache and no plan to build one — fall back to search with include_domains.

wigolo-find-similar