CtrlK
BlogDocsLog inGet started
Tessl Logo

jbaruch/speaker-toolkit

Seven-skill presentation system: ingest talks into a rhetoric vault, run interactive clarification, generate a speaker profile, create presentations that match your documented patterns, produce the deck illustrations + thumbnail visual layer, create and publish talk-content Agent Skills with talk pages to a Jekyll shownotes site, and verify a recorded screencast against its storyboard. Includes a 113-entry Presentation Patterns taxonomy (83 observable: 64 patterns + 19 antipatterns; 30 unobservable: 21 patterns + 9 antipatterns) for scoring, brainstorming, and go-live preparation.

75

Quality

94%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

audit-persisted-pattern-observations.pyskills/vault-ingress/scripts/

#!/usr/bin/env python3
"""Audit a tracking database's persisted pattern observations, read-only.

`persisted_pattern_observations.assess_persisted_pattern_observations` already
classifies one talk, and seven consumers call it in-flow — migration, preflight,
analysis rendering, queue normalization, persistence, the adherence baseline and
the cohort snapshot. Every one of them assesses a talk to decide something about
that talk, then moves on. None of them can answer "what is wrong with this
corpus, in total, before anyone touches it", which is what #167's last
acceptance criterion asks for:

    Run the validator against a copy of the live database and attach the
    deterministic counts to the repair/reparse report.

This is that entry point. It opens a database, assesses every talk, and emits
stable JSON with per-reason-code counts and the affected filenames.

Read-only on purpose. It takes a path and writes nothing, so it is safe to point
at a copy of a live vault — and pointing it at a copy is the intended use, since
the reparse decision wants the counts before the migration runs, not after.

Exit codes follow `audit-pattern-catalog.py`, whose shape this mirrors:

* 0 — every talk's observations are usable.
* 1 — at least one talk is unusable. The report is on stdout; this is a finding
  about the corpus, not a failure of the audit.
* 2 — argparse owns malformed invocations.
* 3 — unexpected failure. One JSON document on stderr, stdout left empty, so a
  caller can tell a broken auditor from a corpus with defects.

A talk carrying no `pattern_observations` at all is reported as
`observations_absent` rather than skipped: on a corpus where 9 of 209 talks had
no block, silence would have read as nine clean talks.

Usage::

    python3 skills/vault-ingress/scripts/audit-persisted-pattern-observations.py \\
        path/to/tracking-database.json
    python3 skills/vault-ingress/scripts/audit-persisted-pattern-observations.py \\
        path/to/tracking-database.json --catalog path/to/patterns
"""

from __future__ import annotations

import argparse
import json
import sys
from collections import Counter
from pathlib import Path
from typing import Any

from failure_diagnostics import emit_unexpected_failure
from persisted_pattern_observations import assess_persisted_pattern_observations
from return_validation import load_catalog
from tracking_database_io import decode_json_object_bytes, snapshot_tracking_database

# Bumped when the emitted shape changes. A consumer that pins a version and
# receives another must refuse the report rather than read fields positionally
# (`rules/stateful-artifacts.md` -> Migration Policy).
REPORT_SCHEMA_VERSION = 1


def audit_database(
    database_path: str | Path,
    catalog_path: str | Path | None = None,
) -> dict[str, Any]:
    """Assess every talk's persisted observations and return the report.

    Separate from `main` so the report can be built and asserted without a
    process boundary, which is what the tests do.
    """
    snapshot = snapshot_tracking_database(database_path)
    database = decode_json_object_bytes(snapshot.raw, snapshot.path)
    catalog = load_catalog(catalog_path)

    talks = database.get("talks")
    if not isinstance(talks, list):
        raise ValueError(
            "tracking database has no talks array; point this at a tracking "
            "database, not a talk file or a profile"
        )

    reason_counts: Counter[str] = Counter()
    # Filenames per reason code, so a count can be acted on rather than only
    # reported. A count alone tells an owner how bad it is and not where.
    reason_filenames: dict[str, list[str]] = {}
    unusable: list[str] = []
    assessed = 0

    for index, talk in enumerate(talks):
        filename = talk.get("filename") if isinstance(talk, dict) else None
        # Index-derived identity for a talk whose own filename is unusable —
        # a malformed record is exactly the kind this audit exists to surface,
        # so it must not be the one entry the report cannot name.
        label = (
            filename if isinstance(filename, str) and filename else f"talks[{index}]"
        )
        assessment = assess_persisted_pattern_observations(talk, catalog)
        assessed += 1
        if assessment.usable:
            continue
        unusable.append(label)
        for code in sorted({finding.reason_code for finding in assessment.findings}):
            reason_counts[code] += 1
            reason_filenames.setdefault(code, []).append(label)

    return {
        "schema_version": REPORT_SCHEMA_VERSION,
        "database": str(Path(database_path).resolve()),
        "summary": {
            "talks_assessed": assessed,
            "talks_usable": assessed - len(unusable),
            "talks_unusable": len(unusable),
        },
        # Sorted so two runs over one database produce byte-identical reports
        # and a diff between two runs is a real change, not dict ordering.
        "reason_counts": dict(sorted(reason_counts.items())),
        "reason_filenames": {
            code: sorted(names) for code, names in sorted(reason_filenames.items())
        },
        "unusable_filenames": sorted(unusable),
        "usable": not unusable,
    }


def _parser() -> argparse.ArgumentParser:
    parser = argparse.ArgumentParser(
        description=(
            "Audit persisted pattern observations in a tracking database "
            "without modifying it."
        ),
    )
    parser.add_argument(
        "database",
        type=Path,
        help="tracking-database.json to audit; safe to point at a copy",
    )
    parser.add_argument(
        "--catalog",
        type=Path,
        default=None,
        help="pattern catalog directory; defaults to the repo's own catalog",
    )
    return parser


def main(argv: list[str] | None = None) -> int:
    """Run the CLI and return its process status."""
    args = _parser().parse_args(argv)
    report = audit_database(args.database, args.catalog)
    # Serialize before writing: a `json.dump` straight to stdout that fails
    # partway leaves a truncated document the caller would try to parse.
    rendered = json.dumps(report, indent=2, sort_keys=True, ensure_ascii=False)
    sys.stdout.write(rendered + "\n")
    if not report["usable"]:
        print(
            f"{report['summary']['talks_unusable']} of "
            f"{report['summary']['talks_assessed']} talks carry unusable "
            "persisted pattern observations; inspect JSON stdout before "
            "migration or reparse",
            file=sys.stderr,
        )
        return 1
    return 0


def run_cli(argv: list[str] | None = None) -> int:
    """Run the CLI behind its failure boundary. Returns the process exit code.

    Importable so the boundary's contract is testable without executing the
    module as a script.
    """
    try:
        return main(argv)
    # The reparse decision reads this report, so a non-zero exit without the
    # stdout document must still say what happened; a traceback would leak vault
    # paths and read an unreadable database as a corpus full of defects.
    except Exception as exc:  # noqa: BLE001 - outer-boundary-process-contract
        emit_unexpected_failure(
            exc,
            "persisted_observation_audit_unexpected_failure",
            "The persisted pattern-observation audit failed unexpectedly. This "
            "command is read-only, so the database is unchanged — but it is "
            "UNAUDITED. Do not begin migration or reparse until a clean run "
            "reports on stdout.",
        )
        return 3


if __name__ == "__main__":
    raise SystemExit(run_cli())

skills

vault-ingress

scripts

adherence_baseline.py

aggregate-catalog-feedback.py

apply-source-repairs.py

artifact_locator.py

artifact_metadata.py

artifact_supervisor.py

audit-pattern-catalog.py

audit-persisted-pattern-observations.py

audit-source-identities.py

batch-download-videos.py

build-contact-sheet.py

build-crop-reviewer.py

build-score-basis.py

catalog_dimension_registry.py

catalog_io.py

catalog_normalization.py

check-runtime.py

classify-pptx-evidence.py

cloud_artifacts.py

cooperative_lock.py

crop_frames.py

crop-reviewer-shell.html

crop-reviewer-shell.html.txt

crop-reviewer.js

crop-reviewer.js.txt

establish-date-provenance.py

failure_diagnostics.py

fetch-transcript.py

ingress_contract.py

local_media_contract.py

local_media_download.py

local_media_evidence.py

local_media_process.py

local_media_sampling.py

local_media_transcription.py

local_media_words.py

markdown_deck.py

migrate-tracking-database.py

mutate-tracking-database.py

pattern_evidence.py

pdf_evidence.py

persist-results.py

persisted_pattern_observations.py

pptx_catalog_selection.py

pptx_deck_facts.py

pptx_discovery_contract.py

pptx_evidence.py

pptx_talk_identity.py

pptx-extraction.py

preflight-vault.py

queue_claim_contract.py

queue-state.py

read-tracking-database.py

render-markdown-deck.py

render-vault-status.py

retained_stage.py

return_validation.py

run-obligations.py

scan-shownotes.py

source_alias_contract.py

source_identity_matching.py

summary_lock.py

sweep-pptx-talk-identity.py

tracking_database_io.py

tracking_database.py

transcript_quality.py

transcript_timing.py

validate-returns.py

vault_root_authority.py

video_evidence.py

video_integrity.py

video-slide-extraction.py

vtt-cleanup.py

write-analysis.py

ytdlp_runtime.py

SKILL.md

README.md

tile.json