CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

pubnub/pubnub-app-developer

Build real-time applications with PubNub pub/sub messaging

Agent Success

Agent success rate when using this tile

94%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.29x

Baseline

Agent success rate without this tile

73%

Overview
Eval results
Skill reviews
Files

rubric.jsonevals/scenario-4/

{
  "context": "Evaluates whether the agent correctly uses PubNub channel groups with proper understanding of limits, publish restrictions, and strategy selection based on channel count thresholds. Tests knowledge of PubNub-specific channel group constraints.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Channel group for scale",
      "description": "The solution uses channel groups (not direct multiplexing of named channels) for subscribing to the large number of channels",
      "max_score": 20
    },
    {
      "name": "2000 per group limit",
      "description": "The solution respects or mentions the 2,000 channels per group limit, splitting across multiple groups if needed",
      "max_score": 20
    },
    {
      "name": "Publish to individual channel",
      "description": "Messages are published to individual channel names, not to a channel group name",
      "max_score": 25
    },
    {
      "name": "10 groups per client",
      "description": "The solution respects or mentions the limit of 10 channel groups per client connection",
      "max_score": 15
    },
    {
      "name": "Guard empty group",
      "description": "The solution includes a check or note that subscribing to an empty channel group causes an error, and ensures channels are added before subscribing",
      "max_score": 20
    }
  ]
}
tessl i pubnub/pubnub-app-developer@0.1.4

evals

SKILL.md

tile.json