CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

pubnub/pubnub-live-auctions

Build real-time auction platforms with PubNub bidding and countdowns

Agent Success

Agent success rate when using this tile

100%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.18x

Baseline

Agent success rate without this tile

85%

Overview
Eval results
Skill reviews
Files

rubric.jsonevals/scenario-1/

{
  "context": "A developer needs to handle the scenario where two bidders submit bids at nearly the same time in a PubNub-based auction, plus ensure that network retries do not cause the same bid to be processed twice.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "KV Store lock acquisition",
      "description": "Acquires a lock in KV Store using a lock-prefixed key (e.g., lock:<auctionId>) before reading auction state",
      "max_score": 15
    },
    {
      "name": "Lock TTL of ~3 seconds",
      "description": "Sets the lock with a short time-to-live (lockedUntil field set to Date.now() plus approximately 2000-5000ms)",
      "max_score": 10
    },
    {
      "name": "Lock release after processing",
      "description": "Releases the lock using kvstore.removeItem after bid processing completes (both accept and reject paths)",
      "max_score": 10
    },
    {
      "name": "Retryable rejection flag",
      "description": "When the lock is already held, rejects the bid with a retryable: true flag in the response message",
      "max_score": 15
    },
    {
      "name": "Idempotency key in message",
      "description": "Bid message includes an idempotencyKey field used for deduplication",
      "max_score": 15
    },
    {
      "name": "Idempotency KV lookup",
      "description": "Checks KV Store for existing result using an idem-prefixed key (e.g., idem:<idempotencyKey>) before processing",
      "max_score": 15
    },
    {
      "name": "Cached result return",
      "description": "If an idempotency key already exists in KV Store, returns the previously cached result instead of reprocessing",
      "max_score": 10
    },
    {
      "name": "Result caching after process",
      "description": "After processing a new bid, stores the result in KV Store keyed by the idempotency key for future deduplication",
      "max_score": 10
    }
  ]
}

evals

scenario-1

rubric.jsontask.md
SKILL.mdtile.json