CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

pubnub/pubnub-live-auctions

Build real-time auction platforms with PubNub bidding and countdowns

Agent Success

Agent success rate when using this tile

100%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.18x

Baseline

Agent success rate without this tile

85%

Overview
Eval results
Skill reviews
Files

rubric.jsonevals/scenario-2/

{
  "context": "A developer needs to implement server-side bid validation for a real-time auction platform using PubNub. The validation must happen before messages reach subscribers, rejecting invalid bids and enriching accepted bids with additional metadata.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Before Publish handler",
      "description": "Uses PubNub Functions Before Publish event handler (not After Publish or On Request) for bid validation",
      "max_score": 15
    },
    {
      "name": "KV Store auction state",
      "description": "Reads current auction state from KV Store using kvstore.get with an auction-keyed lookup (e.g., auction:<id>)",
      "max_score": 10
    },
    {
      "name": "Bid type transformation",
      "description": "Transforms the message type from 'bid' to 'bid_accepted' or 'bid_rejected' by mutating request.message",
      "max_score": 15
    },
    {
      "name": "Auction state check",
      "description": "Validates that the auction state is 'active' or 'closing' before accepting a bid",
      "max_score": 10
    },
    {
      "name": "Minimum amount check",
      "description": "Compares bid amount against currentBid plus minimumIncrement and rejects bids below that threshold",
      "max_score": 10
    },
    {
      "name": "Enrichment fields",
      "description": "Enriched bid_accepted message includes previousBid, previousBidderId, bidNumber, and a server-generated validatedAt timestamp",
      "max_score": 15
    },
    {
      "name": "State update after accept",
      "description": "Updates auction state in KV Store (currentBid, currentBidderId, bidCount) after accepting a valid bid using kvstore.set",
      "max_score": 10
    },
    {
      "name": "Self-outbid prevention",
      "description": "Checks if the bidder is already the current high bidder and rejects with an appropriate reason message",
      "max_score": 10
    },
    {
      "name": "Return request.ok()",
      "description": "Returns request.ok() at the end of all code paths (accept, reject, non-bid passthrough)",
      "max_score": 5
    }
  ]
}

evals

SKILL.mdtile.json