Deterministic tests primary

LLM rubric for quality

Static analysis: ruff

Static analysis: mypy

Static analysis: bandit

State check present

Transcript metrics tracked

Latency metrics tracked

tool_calls not over-specified

Rationale grader ordering

Rationale outcome grading

llm_rubric as primary grader

Natural language assertions

Simulated user persona

Transcript max_turns constraint

State check for outcome

Multi-dimensional success

Transcript metrics tracked

Latency metrics tracked

Design note: persona rationale

Design note: grader complementarity

pass@k definition

pass^k definition

Numeric pass@k example

Numeric pass^k example

Appropriate use of pass@k

Appropriate use of pass^k

Capability vs regression distinction

Saturation response

Balanced problem set guidance

Domain-specific balanced example

LLM judge Unknown option