Build real-time applications with PubNub pub/sub messaging
Agent Success
Agent success rate when using this tile
94%
Improvement
Agent success rate improvement when using this tile compared to baseline
1.29x
Baseline
Agent success rate without this tile
73%
{
"context": "Evaluates whether the agent correctly uses PubNub channel groups with proper understanding of limits, publish restrictions, and strategy selection based on channel count thresholds. Tests knowledge of PubNub-specific channel group constraints.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Channel group for scale",
"description": "The solution uses channel groups (not direct multiplexing of named channels) for subscribing to the large number of channels",
"max_score": 20
},
{
"name": "2000 per group limit",
"description": "The solution respects or mentions the 2,000 channels per group limit, splitting across multiple groups if needed",
"max_score": 20
},
{
"name": "Publish to individual channel",
"description": "Messages are published to individual channel names, not to a channel group name",
"max_score": 25
},
{
"name": "10 groups per client",
"description": "The solution respects or mentions the limit of 10 channel groups per client connection",
"max_score": 15
},
{
"name": "Guard empty group",
"description": "The solution includes a check or note that subscribing to an empty channel group causes an error, and ensures channels are added before subscribing",
"max_score": 20
}
]
}