Build chat applications with PubNub Chat SDK
Agent Success
Agent success rate when using this tile
95%
Improvement
Agent success rate improvement when using this tile compared to baseline
1.83x
Baseline
Agent success rate without this tile
52%
{
"context": "Evaluates whether the agent correctly implements PubNub Chat SDK message reactions and threading using the SDK-specific APIs. Tests knowledge of the toggleReaction pattern and thread lifecycle that are unique to the Chat SDK.",
"type": "weighted_checklist",
"checklist": [
{
"name": "toggleReaction method",
"description": "Reactions are added/removed using message.toggleReaction(emojiName) rather than separate add/remove methods",
"max_score": 20
},
{
"name": "Reactions data structure",
"description": "Reactions are read from message.reactions which returns an object mapping emoji names to arrays of user IDs",
"max_score": 15
},
{
"name": "getThread for existing",
"description": "Existing threads are accessed via message.getThread() to get the thread channel object",
"max_score": 20
},
{
"name": "hasThread check",
"description": "Code checks message.hasThread before attempting to load thread replies",
"max_score": 15
},
{
"name": "thread.sendText for reply",
"description": "Thread replies are sent using thread.sendText(text), not channel.sendText",
"max_score": 15
},
{
"name": "createThread for new",
"description": "When no thread exists yet, the first reply uses message.createThread(text) to create the thread",
"max_score": 15
}
]
}