CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

tessl/pypi-homeassistant

tessl install tessl/pypi-homeassistant@2025.9.0

Open-source home automation platform running on Python 3.

Agent Success

Agent success rate when using this tile

69%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.19x

Baseline

Agent success rate without this tile

58%

rubric.jsonevals/scenario-1/

{
  "context": "This criteria evaluates how effectively the engineer uses Home Assistant's background task management API to implement concurrent device monitoring with proper task tracking and cleanup.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "async_add_executor_job usage",
      "description": "Uses hass.async_add_executor_job() to execute blocking operations (time.sleep()) in the thread pool without blocking the event loop",
      "max_score": 30
    },
    {
      "name": "async_create_background_task usage",
      "description": "Uses hass.async_create_background_task() to create and track background monitoring tasks with appropriate task naming",
      "max_score": 25
    },
    {
      "name": "Task tracking implementation",
      "description": "Maintains references to created background tasks (e.g., in a list or set) to enable proper monitoring and cleanup",
      "max_score": 15
    },
    {
      "name": "Concurrent device monitoring",
      "description": "Implements concurrent monitoring pattern where multiple devices are polled independently without waiting for each other",
      "max_score": 15
    },
    {
      "name": "Task cleanup on stop",
      "description": "Properly cancels and cleans up all tracked background tasks in the stop_monitoring() method",
      "max_score": 10
    },
    {
      "name": "Status data management",
      "description": "Correctly maintains and updates device status data in a thread-safe manner accessible via get_device_status()",
      "max_score": 5
    }
  ]
}

Version

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/homeassistant@2025.9.x
tile.json