CtrlK
BlogDocsLog inGet started
Tessl Logo

getlarge/legreffier-explore

Systematic diary exploration: discover tags, entry distribution, coverage gaps, agent mistakes, and compile recipes

86

1.06x
Quality

90%

Does it follow best practices?

Impact

81%

1.06x

Average score across 5 eval scenarios

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-4/

{
  "context": "Tests whether the agent implements Phase 3 commit pattern analysis: using the most common procedural tag from Phase 1 to find entries, analyzing tag frequency and branch distribution within procedural entries, and detecting anti-patterns like double-prefix tags, missing branch/scope tags, and catch-all tags.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Procedural tag search",
      "description": "Code searches for procedural entries using the most common tag found on procedural entries (e.g., accountable-commit) rather than hardcoding a search query",
      "max_score": 10
    },
    {
      "name": "Fallback search",
      "description": "Code falls back to content-based search with query 'commit' when no obvious procedural tag exists",
      "max_score": 8
    },
    {
      "name": "Tag frequency within procedural",
      "description": "Report shows which tags appear most frequently across procedural entries specifically (not all entry types)",
      "max_score": 8
    },
    {
      "name": "Branch grouping",
      "description": "Report shows branch distribution with counts of procedural entries per branch",
      "max_score": 8
    },
    {
      "name": "Double-prefix detection",
      "description": "Anti-pattern detection catches double-prefix tags like scope:scope:db (entry p4 in sample data)",
      "max_score": 12
    },
    {
      "name": "Missing scope detection",
      "description": "Anti-pattern detection catches entries missing scope tags (entry p5 in sample data)",
      "max_score": 10
    },
    {
      "name": "Missing branch detection",
      "description": "Anti-pattern detection catches entries missing branch tags (entry p7 in sample data)",
      "max_score": 10
    },
    {
      "name": "Catch-all tag detection",
      "description": "Anti-pattern detection identifies overly generic tags that don't follow the namespace convention",
      "max_score": 8
    },
    {
      "name": "Broad entry detection",
      "description": "Anti-pattern detection flags entries with unusually high files-changed counts",
      "max_score": 8
    },
    {
      "name": "Non-procedural filtering",
      "description": "Analysis correctly excludes non-procedural entries (e.g., semantic entry p8) from commit pattern analysis",
      "max_score": 8
    },
    {
      "name": "Report has anti-patterns section",
      "description": "commit-report.md includes a dedicated section listing found anti-patterns with specific entries cited",
      "max_score": 10
    }
  ]
}

evals

tile.json