CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

pubnub/pubnub-chat

Build chat applications with PubNub Chat SDK

Agent Success

Agent success rate when using this tile

95%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.83x

Baseline

Agent success rate without this tile

52%

Overview
Eval results
Skill reviews
Files

rubric.jsonevals/scenario-5/

{
  "context": "Evaluates whether the agent correctly implements PubNub Chat SDK message reactions and threading using the SDK-specific APIs. Tests knowledge of the toggleReaction pattern and thread lifecycle that are unique to the Chat SDK.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "toggleReaction method",
      "description": "Reactions are added/removed using message.toggleReaction(emojiName) rather than separate add/remove methods",
      "max_score": 20
    },
    {
      "name": "Reactions data structure",
      "description": "Reactions are read from message.reactions which returns an object mapping emoji names to arrays of user IDs",
      "max_score": 15
    },
    {
      "name": "getThread for existing",
      "description": "Existing threads are accessed via message.getThread() to get the thread channel object",
      "max_score": 20
    },
    {
      "name": "hasThread check",
      "description": "Code checks message.hasThread before attempting to load thread replies",
      "max_score": 15
    },
    {
      "name": "thread.sendText for reply",
      "description": "Thread replies are sent using thread.sendText(text), not channel.sendText",
      "max_score": 15
    },
    {
      "name": "createThread for new",
      "description": "When no thread exists yet, the first reply uses message.createThread(text) to create the thread",
      "max_score": 15
    }
  ]
}
tessl i pubnub/pubnub-chat@0.1.4

evals

SKILL.md

tile.json