testland/langfuse-tracing
Wires Langfuse tracing into LLM apps for production observability, monitoring, telemetry, and offline eval - instruments via `@observe` (Python) / `startActiveObservation` (TS) decorators that auto-capture inputs / outputs / timings / errors per generation; exposes `langfuse.update_current_span()` for metadata + cost / latency annotation; supports trace-bound scoring for eval datasets and prompt-as-code management. Use when the user needs to monitor, log, trace, or debug LLM API calls in production beyond pre-deploy eval, wants to add LLM observability tooling to an existing app, or wants to ship traces from production to an eval dataset for offline regression testing.
72
91%
Does it follow best practices?
Run evals on this skill
Adds up to 20 points to the overall score
View guide
Low
Low-risk findings worth noting
datasets.mdreferences/
Langfuse datasets for offline eval
Langfuse datasets are collections of (input, expected_output) items.
Build them from three sources:
- Production traces: promote real traces into a dataset for
regression coverage. Sanitize PII before promotion - cross-ref
synthetic-pii-generator. - CSV / JSONL import: load a curated fixture set.
- UI: hand-author items.
Shipping traces from production into a dataset, then replaying them, is
the core offline-regression workflow: item.run() links each replay
back to the dataset so the Langfuse UI can diff the run against a
baseline. See langfuse.com/docs/datasets
for the current API signature.
Validation
After running a dataset, open the Langfuse UI → Datasets → select your
dataset. Each item run should appear under the Runs tab linked to its
trace. If runs are missing, confirm dataset_id is correct and that
item.run() did not raise an exception.