CtrlK
BlogDocsLog inGet started
Tessl Logo

golikovichev/phoenix2pytest

Turn labeled LLM failure traces from an Arize Phoenix project into runnable pytest regression tests using the phoenix2pytest pipeline. Use when the user has an LLM application emitting OpenInference spans to Phoenix and wants a regression suite from real production failures, when extracting test cases from observed LLM bugs (hallucination, format break, off-topic drift, stale data, wrong reasoning, refusal bug), when bridging Phoenix-labeled traces into pytest-based suites for CI, when the user mentions Arize Phoenix MCP, OpenInference instrumentation, LLM observability, Gemini test synthesis, Vertex AI agent evaluation, or wants to react to LLM failures rather than predict them upfront.

88

1.63x
Quality

94%

Does it follow best practices?

Impact

98%

1.63x

Average score across 2 eval scenarios

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

test_form_example.pytests/

"""The landing form ships a ready-to-run example.

A judge opening the hosted demo should be able to click Generate without
hunting for valid input, so the form is pre-filled with an example that
satisfies the same required fields /generate enforces.
"""

from __future__ import annotations

import json

from fastapi.testclient import TestClient

import phoenix2pytest.web as web


def test_example_payload_is_valid_and_complete():
    """The baked-in example must parse and carry the fields /generate requires
    (trace.user_prompt and details.failure_mode), so the default click works."""
    trace = json.loads(web.EXAMPLE_TRACE_JSON)
    details = json.loads(web.EXAMPLE_DETAILS_JSON)
    assert trace.get("user_prompt")
    assert details.get("failure_mode")


def test_form_page_prefills_the_example():
    client = TestClient(web.app)
    body = client.get("/").text
    # The example is rendered into the textareas, not just shown as a placeholder.
    assert "What is the capital of France" in body
    assert "hallucination" in body
    # Style is interpolated (no leftover placeholder token shipped to the browser).
    assert "__STYLE__" not in body
    assert "card" in body

CHANGELOG.md

CONTRIBUTING.md

README.md

REFERENCE.md

SECURITY.md

SKILL.md

tessl.json

tile.json