CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

pubnub/pubnub-chat

Build chat applications with PubNub Chat SDK

Agent Success

Agent success rate when using this tile

95%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.83x

Baseline

Agent success rate without this tile

52%

Overview
Eval results
Skill reviews
Files

rubric.jsonevals/scenario-3/

{
  "context": "Evaluates whether the agent correctly initializes the PubNub Chat SDK using the specific async factory pattern and proper authentication configuration. Tests knowledge of Chat SDK-specific patterns that differ from the base PubNub SDK.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Import from @pubnub/chat",
      "description": "Chat class is imported from '@pubnub/chat' package, not from 'pubnub'",
      "max_score": 20
    },
    {
      "name": "Async Chat.init factory",
      "description": "Initialization uses await Chat.init({...}) async factory method, not a constructor (new Chat(...))",
      "max_score": 25
    },
    {
      "name": "authKey not token",
      "description": "Access Manager credentials are passed via 'authKey' parameter, not 'token' parameter",
      "max_score": 25
    },
    {
      "name": "Persistent userId",
      "description": "userId is provided as a required parameter and is described as persistent/unique per user, not randomly generated",
      "max_score": 15
    },
    {
      "name": "App Context mentioned",
      "description": "The solution mentions or notes that App Context must be enabled in the PubNub Admin Portal keyset settings",
      "max_score": 15
    }
  ]
}
tessl i pubnub/pubnub-chat@0.1.4

evals

scenario-3

rubric.json

task.md

SKILL.md

tile.json