CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

pubnub/pubnub-app-developer

Build real-time applications with PubNub pub/sub messaging

Agent Success

Agent success rate when using this tile

94%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.29x

Baseline

Agent success rate without this tile

73%

Overview
Eval results
Skill reviews
Files

rubric.jsonevals/scenario-2/

{
  "context": "Evaluates whether the agent correctly designs PubNub channel names following naming rules and uses the canonical pattern for direct message channels. Tests knowledge of PubNub-specific channel constraints that differ from generic topic naming.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "No invalid characters",
      "description": "All proposed channel names avoid commas, colons, asterisks, slashes, backslashes, spaces, and periods",
      "max_score": 25
    },
    {
      "name": "DM sorted user IDs",
      "description": "Direct message channel names are constructed by sorting the two user IDs alphabetically so the channel name is identical regardless of who initiates",
      "max_score": 30
    },
    {
      "name": "Case sensitivity noted",
      "description": "The solution acknowledges or accounts for the fact that channel names are case-sensitive (e.g., uses consistent casing convention)",
      "max_score": 15
    },
    {
      "name": "Under 92 characters",
      "description": "Channel names are designed to stay within the 92-character maximum length limit",
      "max_score": 15
    },
    {
      "name": "Descriptive naming",
      "description": "Channel names use a clear prefix-based convention (e.g., 'dm-', 'room-', 'user-') with hyphens or underscores as separators",
      "max_score": 15
    }
  ]
}
tessl i pubnub/pubnub-app-developer@0.1.4

evals

SKILL.md

tile.json