sharaf/llm-learning-system-auditor
Use when the user wants to review, audit, or check safety for an AI memory system, agent learning pipeline, prompt-tuning workflow, skill builder, trace-mining tool, or eval/feedback loop. Produces an evidence-led audit report with learning-loop map, evidence inventory, maturity scorecard, severity-ranked findings, privacy/provenance gaps, counterfactual/eval coverage, and Stabilize/Standardize/Scale roadmap.
100
100%
Does it follow best practices?
Impact
100%
1.28xAverage score across 3 eval scenarios
Passed
No known issues
task.mdevals/scenario-2/
Audit Report: SkillForge Generated Skill Pipeline
Background
SkillForge generates executable agent skills from successful internal sessions. The generated skills can include shell scripts, package installs, and tool permission declarations. The platform team wants to publish SkillForge-generated skills into a shared registry next week.
The team provided only the files in inputs/. You do not have CI logs, package
attestations, reviewer decisions, sandbox policy, production incident data, or
registry moderation records.
Output Specification
Produce a single file called audit_report.md with a complete audit of the
SkillForge system.
Your report should:
- Assess the generated-skill lifecycle and promotion path
- Score maturity with per-domain 0-4 ratings
- Identify safety, provenance, eval, and deployment risks with file and line evidence
- Recommend what must change before shared-registry rollout
Do not treat the presence of a registry manifest as proof that generated skills are safe to execute.