AutoRAG doctor

Goal: every configured datasource is indexed and actually returns hits. A green status is not enough — the run is done only when a real query returns a real hit per datasource, or the datasource is reported as genuinely empty or not configured.

Always finish with the status table in Report.

Safety

• Diagnosis and repair never delete, move, or edit source documents; a doctor run changes only AutoRAG-owned state.
• Never delete a datasource's native store (~/Library/Application Support/katok, ~/.discrawl, ~/.mailcrawl, Telegram/WhatsApp/Notion archives). AutoRAG only reads them; rebuilding them is the owning CLI's job.
• Only AutoRAG-owned state under AUTORAG_HOME and the workspace .autorag directory may be reset.
• Never print token or password values. Report credential names only.
• Kill a process only after confirming it is an AutoRAG-owned orphan.

1. Triage the core

autorag status --json
autorag health --json
autorag setup --format json

• status reports state, stale, diagnostics, and per-component state for minsync, jikji, and datasources. stale: true or any stale-index diagnostic means the corpus changed since the last successful refresh.
• health resolves the single search model and does one live completion probe. --skip-probes only proves config shape, never live access; do not claim a healthy model from it.
• setup --format json reports embedding.backend (remote, daemon, or in-process) and the resolved provider/model/dimension when a backend is alive, or embedding.reason and a remediation when none is. It only probes; it never starts a daemon. Semantic search needs the daemon (or a remote endpoint) to be running: a backend that is absent is a real embedding-unavailable state, not normal.

Config lives at --config, AUTORAG_CONFIG, $AUTORAG_HOME/config.json, or ~/.autorag/config.json. Read searchPaths, workspacePath, minSync, jikji, and datasources before changing anything.

If the CLI itself is missing or the config does not exist, stop and run the autorag-setup skill first — doctor repairs an existing install, it does not create one.

2. Probe every datasource natively

Each CLI owns its archive, so ask the CLI, not AutoRAG. A datasource is only active when its binary exists, its store is present, and its own check passes.

Rules:

• A missing binary is not configured, not a failure. Report it with the install command and move on.
• An empty store is a legitimate zero-hit result. telecrawl and slacrawl return JSON null (not []) for no hits — treat that as empty, not broken.
• A configured datasource whose native check fails is a failure and must be repaired or reported explicitly. Never report a skip as a pass.
• Zero hits with a non-empty store means the CLI's search path is suspect, not the data. Cross-check against the raw index before concluding (for example slacrawl sql "select count(*) from message_fts where message_fts match 'x';" against slacrawl search x). A populated index plus an empty CLI result is an upstream bug: report it with the exact reproduction instead of re-syncing.

3. Prove searchability

Indexing without retrieval is a failed run. Probe retrieval per datasource through the model-free MCP tools, then once end to end:

autorag.status {}
autorag.search {"query":"a word that certainly appears","topK":3}
autorag.search {"query":"recent topic","datasourceIds":["discord"],"topK":3}
autorag.search {"query":"recent mail subject","scope":"/mailcrawl/**","topK":3}

autorag search "summarize the collection" --top-k 3 --json --debug

• Always read the diagnostics returned by MCP autorag.search. A run that silently dropped a whole retrieval method still looks successful, just with fewer results; only the diagnostics name it (retrieval-method-failed). The model-backed CLI autorag search --json hides diagnostics, sessionId, and per-result evidence unless --debug is set, so pass --debug when diagnosing that path.
• A method missing from the returned method values means that method contributed nothing. During a full MinSync re-sync this is expected: the store is being rebuilt, minsync status reports NotSynced, and local-file hits stay absent until it finishes. Confirm with minsync status before treating it as a failure, and never kill a running sync to "fix" it.
• MCP autorag.search needs no model, so it isolates retrieval from model failures.
• Use MCP datasourceIds to select configured connections before retrieval; scope narrows results within scope-capable datasources. Discover connection IDs with autorag.datasources.list; descriptor tags are metadata only, not search filters. Every configured connection is searchable — if one returns nothing, investigate its native store, connector, or the query itself.
• autorag.evidence {"sessionId":"...","resultNumber":N} shows the exact chunk behind a numbered result; use it to confirm a hit is real and its source is readable. The CLI autorag evidence SESSION --json remains for terminal repair.
• Local-file hits must map to an absolute, existing path. Datasource hits keep source-native identities such as /kakao/personal/chunks/42; those are not filesystem paths and must never be passed to cat.

4. Repair playbook

Stuck embedding daemon

AutoRAG owns no lock or pid file for embeddings. A running minsync serve publishes host, port, and pid under ~/.cache/minsync/serve/<instance>/ and removes them on a clean stop. A leftover file from a crashed daemon is harmless: AutoRAG only adopts a daemon whose GET /health answers as a loopback server of the wanted model, so a stale entry is skipped.

ls ~/.cache/minsync/serve/* 2>/dev/null
autorag setup --format json

To diagnose an embedding backend that is not alive, setup --format json and status say so (embedding.reason, and the embedding-unavailable diagnostic). AutoRAG never starts the daemon: run minsync serve --idle-timeout 0 by hand (or as a launchd/systemd service, see docs/minsync-setup.md) and read its own error, or press Ctrl+Shift+E in the TUI. When no daemon answers and the in-process addon (@nomadamas/minsync-native) is missing, that is the embedder-unavailable state. The daemon stops itself after --idle-timeout seconds (default 300), so a semantic failure after a quiet spell usually means it timed out. Stop an orphaned daemon by the pid from ~/.cache/minsync/serve/<instance>/pid after confirming no live refresh holds it.

Index locks are owned by their engines: MinSync/tantivy locks under <workspace>/.autorag/, and per-CLI locks such as <workspace>/.autorag/datasources/discrawl/.discrawl-sync.lock. Delete one only after confirming no owning process is alive; otherwise wait for the run that holds it.

Orphan process

pgrep -fl 'autorag|minsync' | grep -v pgrep

A refresh that was killed mid-run can leave a minsync serve daemon or a minsync child alive. A daemon idle-stops itself; stop a confirmed orphan by the pid from ~/.cache/minsync/serve/<instance>/pid, and only kill a PID directly when it is confirmed orphaned (no parent CLI, no live refresh). Re-run autorag status --json afterwards to confirm inFlight: false.

Embedding model mismatch

embedding-identity-mismatch or a dimension error means the vectors on disk were built with a different embedder than the configured one. Vectors of two different dimensions can never be compared, so the index must be rebuilt:

autorag index rebuild --method minsync

The default is MinSync's local Qwen/Qwen3-Embedding-0.6B engine at 1024 dimensions, reached through a running minsync serve daemon or, failing that, MinSync's in-process engine. Both report one identity (provider: minsync), so a store never re-embeds because the daemon appeared or went away. Vectors written by the removed local Q8_0 gateway engine are not compatible: their identity changed, so the memory store (memory.json.vectors.db), chat-sidecar semantic DBs, and the gist vectors.json re-embed on the next refresh, while a MinSync workspace created through the old gateway profile needs autorag index rebuild --method minsync (or autorag index reset --method minsync --yes then refresh). Changing the embedder's dimension in config without a rebuild leaves retrieval silently empty.

Stale index

stale-index diagnostics or stale: true mean sources changed after the last refresh.

autorag refresh --method parsed,minsync --json
autorag refresh --force --json

Use --force only when incremental refresh does not clear it. For continuous freshness install an hourly autorag watch --once job (cron, launchd, systemd timer, or Task Scheduler).

A file the parser failed on (corrupt PDF, encrypted HWP) or that was unreadable is not re-parsed on every refresh: it is skipped until its mtime, size or parser version changes. After fixing the cause (permissions, OCR setup), run autorag refresh --retry-failed --json to retry those files once without re-indexing healthy ones.

Missing or wrong setup

• unknown-datasource-skill: the config key is not a builtin template and has no "type". It is skipped, not fatal — fix the name or add "type".
• datasource-index-failed / sync-failed: the backing CLI errored. Run that CLI's own check from the table above and fix it there.
• A missing MinSync binary is not a diagnostic: autorag search, autorag refresh and MCP tools fail with MinSync is required ... and a non-zero exit. Install it (cargo install minsync) or leave minSync.autoInstall on, then retry.
• jikji-unavailable: the binary is missing and auto-install failed. It installs through cargo; verify the Rust toolchain, then autorag refresh --method jikji to retry.
• datasource-auth-error / datasource-rate-limited: datasource credentials or quota. Report the missing environment-variable name and let the user supply it.
• A configured datasource that returns nothing is a native store, connector, or query problem — every configured connection is searchable. Run its native check from the table above and fix it there.

Diagnostic codes

Report

Always end with this table, one row per datasource and per AutoRAG component:

Searchable must come from an actual query in step 3, never inferred from index state. Follow the table with the exact remaining action for every row that is not fully green.

Completion condition

Done only when: core triage is clean or every remaining diagnostic is explained, every configured datasource passed its native check, every one of them returned a real hit or is proven empty, every repair was re-verified by re-running the failing check, and the report table was delivered.