Run Skill Insights

Orchestrate the full skill-insights pipeline: discovery → 3 parallel analytical phases → HTML render.

One scan = one report. Whatever the user asks for — "scan this repo", "scan the workspace", "scan A and B and C" — produce ONE consolidated discovery.json + ONE report.html covering all of it. Do NOT run the pipeline once per repo and produce three separate reports — discovery's --repo flag is designed exactly so you can scan multiple repos in a single run.

Pipeline shape

discover-skills → discovery.json
                     ↓
       ┌────────┬────┴────┬────────────┐
       ↓        ↓         ↓            ↓
   staleness  quality  duplicates  registry-search
   (script) (script+LLM)(script+LLM)  (script+HTTP)
       ↓        ↓         ↓            ↓
       └────────┴────┬────┴────────────┘
                     ↓
                render report

Each analytical phase reads discovery.json independently and writes its own output. They never share state. The render step reads all five (discovery, staleness, quality, duplicates, registry-search) and produces a unified HTML report.

Step 0: Interpret what the user asked for

This step is critical and easy to get wrong. Map the user's request to ONE of three scan modes:

Do NOT dispatch separate orchestrator instances per repo. The whole point of the explicit-selection mode is that a single scan handles multiple repos and produces ONE consolidated report. If the user names three repos, you call discovery once with three --repo flags, then run the analytical phases ONCE against the combined discovery.json.

If you're unsure whether the user wanted one report or three, ask. Default assumption: one consolidated report.

Inputs

• SCAN_ROOT — the directory to scan. Defaults to $(pwd). For explicit-selection mode, use the common parent of the named repos (e.g. ~/repos/).
• REPOS — optional list of repo paths for explicit-selection mode. Each becomes a --repo PATH flag passed to discovery.
• OUTPUT_DIR — where to write outputs. Defaults to $SCAN_ROOT/.skill-insights/. The whole pipeline writes here.

Step 1: Setup

SCAN_ROOT="${SCAN_ROOT:-$(pwd)}"
OUTPUT_DIR="$SCAN_ROOT/.skill-insights"
mkdir -p "$OUTPUT_DIR"
SCAN_ID="scan-$(date +%Y%m%d-%H%M%S)"

if [ -d "$HOME/.tessl/tiles/tessleng/skill-insights/references" ]; then
  TILE_ROOT="$HOME/.tessl/tiles/tessleng/skill-insights"
elif [ -d "$(pwd)/.tessl/tiles/tessleng/skill-insights/references" ]; then
  TILE_ROOT="$(pwd)/.tessl/tiles/tessleng/skill-insights"
else
  echo "ERROR: skill-insights tile not found." >&2; exit 1
fi

Step 2: Discovery

Invoke discover-skills (see its SKILL.md). It writes $OUTPUT_DIR/discovery.json.

Pass --repo PATH repeatedly when the user named a specific set of repos:

# Single repo (or workspace auto-discovery)
python3 "$TILE_ROOT/skills/discover-skills/scripts/discover_skills.py" \
  --scan-root "$SCAN_ROOT" \
  --output "$OUTPUT_DIR/discovery.json"

# Explicit selection: scan repos A, B, C in one combined run
python3 "$TILE_ROOT/skills/discover-skills/scripts/discover_skills.py" \
  --scan-root "$SCAN_ROOT" \
  --repo "$SCAN_ROOT/repo-a" \
  --repo "$SCAN_ROOT/repo-b" \
  --repo "$SCAN_ROOT/repo-c" \
  --output "$OUTPUT_DIR/discovery.json"

Wait for it to complete. Validate:

[ -f "$OUTPUT_DIR/discovery.json" ] && jq -e '.schema_version == "1.3"' "$OUTPUT_DIR/discovery.json" > /dev/null

If discovery fails, stop. The downstream phases all depend on discovery.json.

Step 3: Run 4 analytical phases in parallel

Once discovery.json exists, dispatch the four phases as parallel subagents. Each phase reads discovery.json independently and writes its own output JSON. Quality and duplicates have absorbed their LLM work into single Python scripts (quality shells out to tessl skill review; duplicates still uses prep + per-pair subagents — see below) — so from the orchestrator's perspective, all four phases look uniform.

In a single assistant message, invoke the Task tool four times:

Subagent A — staleness + git provenance

Skill: analyze-skill-staleness. Reads discovery.json, runs a single git log per skill, derives age signals + per-skill git_provenance (created_by, last_modified_by, contributors, recent commits) from one stream, writes staleness.json (schema 1.1). Fully deterministic — no LLM dispatch.

Pass to the subagent:

• The discovery path: $OUTPUT_DIR/discovery.json
• The output path: $OUTPUT_DIR/staleness.json
• The skill SKILL.md path so it knows where the bundled script lives

Subagent B — quality

Skill: analyze-skill-quality. Reads discovery.json, then runs tessl skill review --json per skill in parallel batches (concurrency 8). Tile-level quality is pulled directly from discovery.tiles[].registry.scores.quality for tiles already scored on the registry. The orchestrating subagent runs ONE script (analyze_quality.py) which emits quality.json directly — no per-skill prompt files, no per-skill verdict dispatch, no finalize step.

Pass to the subagent:

• The discovery path: $OUTPUT_DIR/discovery.json
• Final output: $OUTPUT_DIR/quality.json
• Optional: --skip-published-skills to avoid tessl skill review calls for skills whose owning tile already has a registry.scores.quality (uses the tile-level score as a passthrough; trades per-skill detail for speed)

Subagent C — duplicates

Skill: detect-skill-duplicates. Reads discovery.json, runs prep script to compute candidate pairs (default cap: 10 highest-similarity pairs) and write a duplicates-prompts/ directory (one prompt file per pair + an index.json), dispatches all sub-subagents in a single parallel batch (each writing its verdict to duplicates-verdicts/<idx>.json), then runs the finalize script.

Pass to the subagent:

• The discovery path: $OUTPUT_DIR/discovery.json
• Prompts directory: $OUTPUT_DIR/duplicates-prompts/
• Verdicts directory: $OUTPUT_DIR/duplicates-verdicts/
• Final output: $OUTPUT_DIR/duplicates.json

Subagent D — registry-search

Skill: registry-search. Reads discovery.json, filters to skills with source_type: "standalone", queries /experimental/search (hybrid mode) once for the top-1 skill match and once for the top-1 tile match per candidate, picks the higher-aggregate-score hit as best_match, and writes registry-search.json (schema 1.2). Stdlib + HTTP only — no LLM, no agent judgement.

Pass to the subagent:

• The discovery path: $OUTPUT_DIR/discovery.json
• Final output: $OUTPUT_DIR/registry-search.json
• Optional: --registry-base-url, --concurrency (defaults: https://api.tessl.io, 8). The endpoint is queried anonymously — no Tessl auth required.

Wait for all four

You MUST wait for every subagent to return before proceeding to render. Each one writes its output to disk; the render step reads from disk.

If any phase fails:

• Staleness fails: rare (deterministic script). If it errors, surface stderr and proceed without staleness. The report renders without that section.
• Quality fails: usually means tessl skill review is unavailable, the user isn't authenticated, or the registry is unreachable. The phase records failed_skills[] in metadata; quality.json still gets written with whatever succeeded. Proceed.
• Duplicates fails: typically a subagent-batch failure. The phase records failed_pairs[] in metadata. Duplicates.json still gets written. Proceed.
• Registry-search fails: typically a missing auth token or the registry being unreachable. The phase still writes a valid registry-search.json with per-record search_errors[] and a top-level warnings[] notice; the renderer treats it as "skipped". Proceed.

After all return, verify:

ls -la "$OUTPUT_DIR"/{discovery,staleness,quality,duplicates,registry-search}.json 2>/dev/null
jq -e '.schema_version == "1.3"' "$OUTPUT_DIR"/discovery.json > /dev/null
jq -e '.schema_version == "1.1"' "$OUTPUT_DIR"/staleness.json > /dev/null
jq -e '.schema_version == "2.0"' "$OUTPUT_DIR"/quality.json > /dev/null
jq -e '.schema_version == "1.0"' "$OUTPUT_DIR"/duplicates.json > /dev/null
jq -e '.schema_version == "1.2"' "$OUTPUT_DIR"/registry-search.json > /dev/null

Each phase script also self-validates its output against the corresponding JSON Schema in references/schemas/ before writing (when jsonschema is installed). A schema-mismatched output exits with code 2 before reaching disk, so by the time you read these files they're guaranteed to conform to the schema versions above. Missing-file is acceptable for the render step (the report degrades gracefully); malformed JSON or a wrong schema_version is not.

Step 4: Render HTML report

Read the HTML report template. It now has multiple injection points:

• <!--@DISCOVERY_DATA@--> — the full discovery.json contents
• <!--@STALENESS_DATA@--> — the full staleness.json contents (or null if missing)
• <!--@QUALITY_DATA@--> — the full quality.json contents (or null if missing)
• <!--@DUPLICATES_DATA@--> — the full duplicates.json contents (or null if missing)
• <!--@REGISTRY_SEARCH_DATA@--> — the full registry-search.json contents (or null if missing)

Substitute each placeholder. Write to $OUTPUT_DIR/report.html. Prefer the bundled renderer so template placeholders and phase filenames stay in one tested place:

python3 "$TILE_ROOT/skills/run-skill-insights/scripts/render_report.py" \
  --template "$TILE_ROOT/references/report-template.html" \
  --output-dir "$OUTPUT_DIR" \
  --report "$OUTPUT_DIR/report.html"

Equivalent implementation sketch:

TEMPLATE="$TILE_ROOT/references/report-template.html"
REPORT="$OUTPUT_DIR/report.html"

python3 <<'PY'
import json, os
def load_or_null(p):
    if not os.path.exists(p): return "null"
    return open(p).read().replace('</', '<\\/')

template = open(os.environ['TEMPLATE']).read()
out = template
out = out.replace('<!--@DISCOVERY_DATA@-->', load_or_null(os.environ['OUTPUT_DIR'] + '/discovery.json'))
out = out.replace('<!--@STALENESS_DATA@-->', load_or_null(os.environ['OUTPUT_DIR'] + '/staleness.json'))
out = out.replace('<!--@QUALITY_DATA@-->', load_or_null(os.environ['OUTPUT_DIR'] + '/quality.json'))
out = out.replace('<!--@DUPLICATES_DATA@-->', load_or_null(os.environ['OUTPUT_DIR'] + '/duplicates.json'))
out = out.replace('<!--@REGISTRY_SEARCH_DATA@-->', load_or_null(os.environ['OUTPUT_DIR'] + '/registry-search.json'))
open(os.environ['REPORT'], 'w').write(out)
PY

# Verify no placeholders remain
! grep -q '<!--@.*_DATA@-->' "$REPORT" || echo "ERROR: placeholder not substituted"

Step 5: Surface results to the user

Skill Insights scan complete.

Scan root:        <scan_root>
Repos:            <N>
Skills:           <N> logical (<M> paths)
Tiles:            <N> total — published_to_registry=<N>, authored_only=<N>
                  by_tier: published=<N>, authored=<N>, github=<N>
Security:         <N> tiles flagged MEDIUM/HIGH/CRITICAL
Updates:          <N> tiles outdated vs registry

Staleness:        median <D>d, <stale> stale, <broken> with broken refs
Quality:          avg <S>/100, <good>/<accept>/<needs_work>/<poor>
                  (<R> from registry, <S> via tessl skill review, <P> passthrough)
Duplicates:       <C> clusters covering <N> skills (potential reduction: <R>)
Registry:         <K> skill matches, <T> tile matches across <N> non-published skills

Context budget:   <FL> tokens front-loaded, <ODmin>-<ODmax> on-demand range

JSON:             <OUTPUT_DIR>/{discovery,staleness,quality,duplicates,registry-search}.json
Report:           <OUTPUT_DIR>/report.html

Open the report:
  open <OUTPUT_DIR>/report.html

Read the four JSON files to populate the summary; gracefully omit any phase that didn't produce output.

Step 6: Done

End. The user has a JSON inventory + per-skill staleness scores + per-skill quality scores + duplicate clusters + per-skill registry-search suggestions + a unified HTML report.

Future phases (not yet built)

Behaviour analysis (load-bearing skills from local agent conversation logs), security (Snyk integration), missing-skills (workflow-gap analysis from logs). Each would slot in as another parallel branch alongside the current three. The render step's template would gain a new injection point per phase.

A sibling skill, posthog-skill-query, partially addresses what an "activation health" phase would do — it pulls org-wide skill / loaded-skill / MCP usage from PostHog's cli:agent-signals:* event stream. It's deliberately standalone and not invoked by this orchestrator; it writes its own org_usage.json + org_usage.html for downstream cross-reference. Run it independently when you want that view.