CtrlK
BlogDocsLog inGet started
Tessl Logo

golikovichev/pytest-conversational

Test chat bots, voice assistants, and IVR menus with pytest using a small Conversation object and a callable bot adapter. Use when the user wants to write rule-based assertions over multi-turn dialogue without bringing in an LLM dependency, when they have a chatbot reachable as a Python callable or HTTP webhook, when they need to keep per-conversation state across turns and assert on slot filling, when they want pytest-native fixtures and a printable transcript on failure, or when they mention voice-assistant testing, IVR menu testing, conversational AI testing, LLM bot testing (used as the target under test, not as the matcher), expect matchers for bot replies, or multi-turn dialogue tests.

99

1.56x
Quality

100%

Does it follow best practices?

Impact

97%

1.56x

Average score across 3 eval scenarios

SecuritybySnyk

Passed

No known issues

Overview
Quality
Evals
Security
Files

task.mdevals/scenario-1/

Pizza Ordering Bot Test Suite

Problem Description

The engineering team at PizzaRapido has built a conversational pizza ordering bot. The bot guides customers through a three-step ordering process, collecting size, crust type, and toppings across separate turns before confirming the order. Throughout the conversation, the bot classifies each customer message with an intent label (e.g., "select_size", "select_crust", "select_topping", "confirm_order") and records which slot was extracted from that turn. The bot also tracks response latency so the team can enforce a performance SLA.

Your job is to write a pytest test suite using pytest-conversational that validates the bot's multi-turn slot-filling behaviour end to end. The bot should be implemented as a pure Python callable directly in the test file — no external services required. You need to verify that the bot correctly fills the size, crust, and topping slots, that state transitions work across turns, that per-turn intent and slot metadata is recorded correctly, and that the bot replies within an acceptable time window.

Install the library with pip install pytest-conversational before writing the tests. Python 3.10+ is required.

Output Specification

Produce the following files:

  • test_pizza_bot.py — the pytest test file containing the bot implementation (as a Python callable) and the full test suite.
  • results.txt — the captured output of running pytest test_pizza_bot.py --tb=short -v, saved by piping or redirecting pytest output to this file.

The test suite should include at minimum:

  • A test that drives a complete three-turn ordering sequence and checks that all slots are filled by the end.
  • A test that verifies the bot's reply on each turn (or at the last turn) contains an appropriate confirmation message.
  • Tests that assert on per-turn metadata (intent and extracted slot) using the library's metadata assertion helpers.
  • A test that checks that conversation-level state reflects the collected order at the end of the dialogue.
  • A test that the bot's response time on each turn is within an acceptable budget.

CHANGELOG.md

CONTRIBUTING.md

README.md

REFERENCE.md

SECURITY.md

SKILL.md

tessl.json

tile.json