Build chat applications with PubNub Chat SDK
Agent Success
Agent success rate when using this tile
95%
Improvement
Agent success rate improvement when using this tile compared to baseline
1.83x
Baseline
Agent success rate without this tile
52%
{
"context": "Evaluates whether the agent correctly initializes the PubNub Chat SDK using the specific async factory pattern and proper authentication configuration. Tests knowledge of Chat SDK-specific patterns that differ from the base PubNub SDK.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Import from @pubnub/chat",
"description": "Chat class is imported from '@pubnub/chat' package, not from 'pubnub'",
"max_score": 20
},
{
"name": "Async Chat.init factory",
"description": "Initialization uses await Chat.init({...}) async factory method, not a constructor (new Chat(...))",
"max_score": 25
},
{
"name": "authKey not token",
"description": "Access Manager credentials are passed via 'authKey' parameter, not 'token' parameter",
"max_score": 25
},
{
"name": "Persistent userId",
"description": "userId is provided as a required parameter and is described as persistent/unique per user, not randomly generated",
"max_score": 15
},
{
"name": "App Context mentioned",
"description": "The solution mentions or notes that App Context must be enabled in the PubNub Admin Portal keyset settings",
"max_score": 15
}
]
}