Context Index

Scan, validate, and index every context file the project has captured.

Mindset

The index is a cache, never a ledger. The source of truth for what exists is always the YAML frontmatter inside each context file; the generated index is a derived summary that lets an agent answer "what plans/findings/decisions exist?" without reading every file. Because it is derived, regeneration is idempotent and safe to run whenever there is doubt about freshness — there is no state to corrupt, only a file to rewrite from scratch.

A file with missing or malformed frontmatter is not a formatting nitpick: it is invisible to every tool and agent that reads the index instead of the raw directory tree. Treat exclusion-from-index as the real failure mode, not the warning text that reports it. Balance that with proportionate language — flag the gap clearly (ALWAYS regenerate after a change, NEVER hand-edit the index) without turning every missing tag into a blocking emergency.

Prerequisites

• One or more context files, each with frontmatter that is valid or fixable.
• The create-context-file skill (companion to this one) to add or repair frontmatter on files that lack it, using its typology set and templates.
• Shell access (bash, python3, git) to run this skill's scripts.

When to Use

• After creating, renaming, or deleting a context file.
• When a pre-commit or CI gate blocks with a message like "missing YAML frontmatter" or "index is stale."
• When asked "what plans / findings / decisions / follow-ups exist?" — read the index rather than walking the directory tree by hand.
• After a bulk import, migration, or retroactive frontmatter cleanup touching many context files at once.

When Not to Use

• Do not run this as a substitute for creating a missing file — create it first with the create-context-file skill, which owns the filename and frontmatter shape.
• Do not edit the generated index by hand — it is fully regenerated and any manual edit is silently overwritten on the next run.
• Do not treat a clean regeneration as proof every context file is meaningful or current — it only proves the frontmatter parses; stale content still needs a human to retire it.

Frontmatter Schema

Every context file is expected to carry this block (the same shape create-context-file writes):

---
title: "Human-readable title"
type: finding | plan | guide | follow-up | merge-request | ticket | decision | note | research
date: YYYY-MM-DD
status: active | done
tags: []
related:
  - ../relative/path/to/related.md # omit the whole key if there is nothing related
blocks:
  - ../relative/path/to/dependent.md # omit if this file blocks nothing
blocked-by:
  - ../relative/path/to/blocker.md # omit if nothing blocks this file
severity: CRITICAL | HIGH | MEDIUM | LOW # follow-ups only; set with priority and graded, last
priority: P1 | P2 | P3
graded: YYYY-MM-DD # the day the user confirmed the two grades
---

type: is the singular form of the plural typology folder a file lives under (findings/ → finding, follow-ups/ → follow-up, research/ → research). The index groups entries back into the plural form for display. title, type, status, and date are required for a file to appear in the index; tags, related, blocks, and blocked-by are optional and only rendered when non-empty. blocked-by is read by scripts/context-ready.sh to compute which active files have zero open blockers.

Expect a follow-up's severity, priority and graded (the rubric lives in the create-context-file skill) in its index entry, after date:, whenever the file sets them. Use them to sort by grade without opening each file. Fix the source file when stderr reports a value outside the lists or an incomplete set: the index still writes it as authored, so readers show it as needing a grade.

Procedure

1. Check for missing or malformed frontmatter first, running scripts/check-context-frontmatter.sh against every context file. — Verify: the script prints nothing and exits 0. If it lists files, those are invisible to the index until fixed.
2. Fix anything the check flagged, using the create-context-file typology set and the schema above — not by inventing an ad hoc shape.
3. Validate against the full schema when precision matters (e.g. before a CI gate), running scripts/validate-context-frontmatter.sh against the changed files — this also catches an out-of-enum type or status value that the looser check above would miss.
4. Regenerate the index, running scripts/regenerate-context-index.sh. — Stop if: it reports files excluded on stderr; go back to step 2 rather than treating the run as complete.
5. Stage the regenerated index alongside any context-file change in the same commit.

Quick Commands

# Full session: check, fix, regenerate, stage
./scripts/check-context-frontmatter.sh .context/**/*.md
# (fix anything reported, using create-context-file's schema)
./scripts/regenerate-context-index.sh
git add .context/index.yaml

Expected result: a line reporting how many entries were generated and where the index was written, with zero stderr warnings.

# Strict schema validation (enum + date-pattern checks) before a CI gate
./scripts/validate-context-frontmatter.sh .context/**/*.md

Expected result: OK: <N> file(s) validated against schema, or a list of per-file errors with exit code 1.

# Enforce the date-first filename convention on known typology directories
./scripts/check-context-filenames.sh

Expected result: context filenames OK, or a list of naming violations.

# Advisory: surface active plans/follow-ups that have gone stale
./scripts/check-plan-staleness.sh

Expected result: a notice listing entries older than the threshold (default 60 days via CONTEXT_STALENESS_THRESHOLD_DAYS), or no output. Always exits 0 — this is a nudge, not a gate.

# What's ready to work on right now, computed from blocked-by
./scripts/context-ready.sh

# What's blocked, and by what
./scripts/context-ready.sh --blocked

Expected result: a list of active files with zero open blockers (or a notice that none are), read straight from the index — no mutation, always exits 0. A blocker is "open" when the file it points at is missing from the index or not status: done; an unresolvable reference fails closed rather than being treated as satisfied. Run regenerate-context-index.sh first if the index might be stale (this script warns, but does not regenerate, on a mismatch).

Split index (opt-in)

index.yaml is one large file. --split also writes one pair per non-empty typology beside it in .context/: index-<type>.yaml (same section: key and entry shape as index.yaml) and index-<type>.md (a table of linked title, status and date; follow-ups add severity, priority and graded). The per-type headers carry no date, so a day passing does not rewrite them.

./scripts/regenerate-context-index.sh --split                 # index.yaml and the pairs
./scripts/regenerate-context-index.sh --split --no-aggregate  # the pairs only
./scripts/regenerate-context-index.sh --split --check         # stale, missing and orphan files fail
./scripts/regenerate-context-index.sh --only plans --stdout   # render one file, write nothing
./scripts/regenerate-context-index.sh --only plans --stdout --format md

Without --split the output is unchanged. A type that empties has its pair deleted on the next --split run, and --split --check fails on a leftover file or a missing one. The index- prefix is reserved: every script here skips index-*.md by name, and create-context-file refuses a typology or slug that starts with it.

Reader rule

Every reader applies the same rule: if any index-<type>.yaml exists, the index is split, and a type with no file is empty. Only when none exists is index.yaml read. context-ready.sh and check-plan-staleness.sh follow it (the split files as one set, because blockers cross types), and context-ready.sh warns when the sum of the split files and index.yaml disagree.

A stale split file beats a fresh index.yaml. Once split files exist, a plain regenerate-context-index.sh run refreshes only index.yaml, which readers then ignore. Regenerate with --split (or --split --no-aggregate), or delete the split files to go back to the single index. Never leave both generations to drift.

Anti-Patterns

NEVER edit the generated index by hand

WHY: it is fully regenerated by regenerate-context-index.sh; a manual edit is overwritten on the very next run and gives a false sense of correctness in the meantime.

BAD: adding or editing an entry directly in the generated index file. GOOD: update the source file's frontmatter, then re-run regenerate-context-index.sh.

Consequence: the index silently drifts from the files it claims to summarize, and the next regeneration erases the "fix" without warning.

NEVER ignore stderr warnings from regenerate-context-index.sh

WHY: files excluded from the index are invisible to any agent or tool that reads the index instead of the raw directory — they cannot be found, followed up on, or superseded.

BAD: running the script, seeing warnings, and treating stdout's success line as the whole story.

GOOD: fix every warning first, then regenerate, then treat the output as complete.

Consequence: an active plan or a critical finding quietly disappears from every summary an agent produces from the index.

NEVER rely on the index as the sole inventory of context files

WHY: a file with missing or malformed frontmatter is excluded, not flagged inline — the index only reflects files that already parse.

BAD: answering "what context files exist?" purely from the generated index without having run the frontmatter check first.

GOOD: run check-context-frontmatter.sh, resolve what it reports, then treat the freshly regenerated index as reliable.

Consequence: a stakeholder is told a decision or follow-up doesn't exist when it does — it is simply unindexed.

NEVER let a file's type: disagree with the directory it lives in

WHY: the directory already declares the typology; a mismatched type: value is a mistake in the file, not a new category to route around silently.

BAD: a file under a plans/ directory carrying type: finding, quietly folded into an other catch-all that most tooling never reads.

GOOD: fix the file's type: to match its directory, or move the file to the directory that matches its real type.

Consequence: the regenerated index groups the entry somewhere nobody looks, which is functionally the same as it not existing.

References