CtrlK
BlogDocsLog inGet started
Tessl Logo

jbaruch/hubitat-dev

Context for developing and debugging Hubitat Elevation apps, drivers, and hub environment — sandbox constraints, lifecycle idioms, capability contracts, plus grounded deploy/log-tail/lint mechanisms.

81

Quality

94%

Does it follow best practices?

Impact

27%

Average score across 2 eval scenarios

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

criteria.jsonevals/scenario-2/

{
  "context": "Grades exact knowledge of the Hubitat sandbox import allow-list (plugin ships reference/allowed-imports.txt + the sandbox-constraints rule), weighted toward the COUNTERINTUITIVE cases where a baseline agent reliably guesses wrong. Verified truth: ALLOWED = java.util.ArrayList, groovy.json.JsonSlurper, java.util.concurrent.ConcurrentHashMap; REJECTED = java.io.File, java.net.HttpURLConnection, java.lang.Thread. The two obvious-allowed cases (ArrayList, JsonSlurper) are NOT scored — baseline gets them free, and scoring them inflates the baseline. Each criterion below is a single yes/no fact so the scorer can credit a concise per-import verdict; the earlier version bundled multiple imports per criterion and scored 0 because the scorer could not award partial credit.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "allows_concurrenthashmap",
      "description": "Classifies java.util.concurrent.ConcurrentHashMap as ALLOWED. This is the most counterintuitive case — a concurrency primitive feels sandbox-hostile but it is on the allow-list. FAIL (0) if called rejected/blocked.",
      "max_score": 30
    },
    {
      "name": "rejects_http_url_connection",
      "description": "Classifies java.net.HttpURLConnection as REJECTED. Baseline commonly assumes a standard HTTP class is fine. FAIL (0) if called allowed.",
      "max_score": 25
    },
    {
      "name": "rejects_java_io_file",
      "description": "Classifies java.io.File as REJECTED. FAIL (0) if called allowed.",
      "max_score": 20
    },
    {
      "name": "rejects_thread",
      "description": "Classifies java.lang.Thread as REJECTED. FAIL (0) if called allowed.",
      "max_score": 10
    },
    {
      "name": "names_native_alternatives",
      "description": "For the rejected HttpURLConnection and File, names the Hubitat-native replacement: httpGet/asynchttpGet for HTTP, and File Manager or state for file storage. Award ~7 for one, 15 for both; 0 if no alternative is named.",
      "max_score": 15
    }
  ]
}

.mcp.json

README.md

tile.json