CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

pubnub/pubnub-live-auctions

Build real-time auction platforms with PubNub bidding and countdowns

Agent Success

Agent success rate when using this tile

100%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.18x

Baseline

Agent success rate without this tile

85%

Overview
Eval results
Skill reviews
Files

rubric.jsonevals/scenario-3/

{
  "context": "A developer needs to set up the foundational channel architecture and auction lifecycle management for a PubNub-based auction platform, including proper naming patterns and state transitions.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Dot-separated channels",
      "description": "Uses dot-separated hierarchical channel names (e.g., auction.item-5001, not auction/item-5001 or auction_item-5001)",
      "max_score": 10
    },
    {
      "name": "Per-auction bid channel",
      "description": "Defines a per-auction channel in the format auction.<itemId> for live bid updates",
      "max_score": 10
    },
    {
      "name": "Activity sub-channel",
      "description": "Defines an activity sub-channel in the format auction.<itemId>.activity for bid history feed",
      "max_score": 10
    },
    {
      "name": "Admin sub-channel",
      "description": "Defines an admin sub-channel in the format auction.<itemId>.admin for administrative controls",
      "max_score": 10
    },
    {
      "name": "Catalog channels",
      "description": "Uses catalog.active, catalog.upcoming, and/or catalog.completed channels for auction listings",
      "max_score": 10
    },
    {
      "name": "Notification channel",
      "description": "Uses user.<userId>.notifications channel pattern for personal bid alerts",
      "max_score": 10
    },
    {
      "name": "All lifecycle states",
      "description": "Defines at least these auction states: scheduled, active, closing, completed, paused, cancelled",
      "max_score": 15
    },
    {
      "name": "Closing state at 60s",
      "description": "The 'closing' state is entered when 60 seconds or less remain, with countdown broadcast frequency increasing to every 1 second from every 10 seconds",
      "max_score": 15
    },
    {
      "name": "State transitions defined",
      "description": "Defines valid state transitions (e.g., scheduled->active, active->closing, closing->completed, active->paused)",
      "max_score": 10
    }
  ]
}

evals

SKILL.mdtile.json