CtrlK
BlogDocsLog inGet started
Tessl Logo

vibe-ai/spec-as-source

Adds spec-as-source enforcement to any project using spec-driven development

87

2.34x
Quality

91%

Does it follow best practices?

Impact

82%

2.34x

Average score across 5 eval scenarios

SecuritybySnyk

Passed

No known issues

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-2/

{
  "context": "The agent is asked to run spec-ci-sync. The project has pytest.ini present and a spec manifest listing tests/auth/test_login.py and tests/auth/test_rate_limit.py as spec-declared test files.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "detects-pytest",
      "description": "Detects pytest as the test runner from the presence of pytest.ini",
      "max_score": 2
    },
    {
      "name": "reads-manifest",
      "description": "Reads .spec-source-manifest.json or runs build-spec-manifest.py to collect test files",
      "max_score": 1
    },
    {
      "name": "identifies-test-files",
      "description": "Identifies tests/auth/test_login.py and tests/auth/test_rate_limit.py as spec-declared tests",
      "max_score": 2
    },
    {
      "name": "generates-pytest-workflow",
      "description": "Generates a workflow with actions/setup-python and pytest steps",
      "max_score": 2
    },
    {
      "name": "includes-test-files-in-cmd",
      "description": "Includes the spec-declared test files in the pytest command",
      "max_score": 2
    },
    {
      "name": "includes-spec-checks",
      "description": "Includes all three spec check steps: check-spec-links, check-target-ownership, build-spec-manifest",
      "max_score": 2
    },
    {
      "name": "shows-diff-and-confirms",
      "description": "Shows a diff before writing the workflow and asks for confirmation",
      "max_score": 1
    },
    {
      "name": "no-extra-runners",
      "description": "Does not add test runner steps for runners not present in the project",
      "max_score": 1
    }
  ]
}

README.md

tile.json