Cekura Predefined Metrics

Purpose

Predefined metrics are Cekura's built-in evaluators — ready to enable on any agent with no prompt writing required. They cover the most common quality dimensions across accuracy, conversation quality, customer experience, and speech quality. Use this skill to decide which predefined metrics to enable and how to configure them.

Performing Platform Actions

When this skill suggests creating, listing, updating, or evaluating something on Cekura, prefer using available platform tools over describing API calls or dashboard steps. In Claude Code with the Cekura plugin installed, these tools are auto-configured and handle authentication, parameter validation, and error handling for you. Fall back to direct API endpoints or dashboard guidance only when no tools are available in the current session.

Core Terminology

• Main agent: The client's AI voice agent being tested
• Testing agent: Cekura's simulated caller that exercises the main agent
• Predefined metric: Built-in evaluator shipped by Cekura — no prompt required, identified by a code
• Custom metric: User-authored metric with a custom prompt or custom_code (see cekura-metric-design)
• Simulation: Test runs using Cekura's testing agent against the main agent (Sim column in the catalog)
• Observability: Real production calls flowing through the agent (Obs column in the catalog)
• Project-level toggle: Enables a predefined metric across simulation OR observability for an entire project
• Evaluator attachment: Adds the metric to a specific test scenario; required for the metric to fire on that evaluator

Predefined vs Custom Metrics

Enable predefined metrics first. They require zero prompt engineering and cover the most common quality dimensions out of the box. Only reach for a custom metric when:

• You need to evaluate a business-specific workflow (booking flow, escalation protocol, etc.)
• You need to check agent behavior against your specific system prompt
• You need to combine multiple signals into one score

For everything else, a predefined metric will give you reliable, consistent results faster.

The Predefined Metrics Workflow

1. Browse the catalog — Use the four catalog tables below to identify candidate metrics. Filter by Sim/Obs availability, cost, and required constraints.
2. Pick a starting set — Begin with the Baseline below. For richer coverage, see references/selection-by-use-case.md for recommended sets per agent type (booking, collections, support, healthcare, voice-quality investigation).
3. Toggle on at the project level — Enables the metric for simulation runs (or observability — they are independent toggles). Without this, attaching the metric to an evaluator does nothing.
4. Add to individual evaluators — Attaches the metric to specific test scenarios so it fires when that scenario runs.
5. Configure if required — Six metrics need or accept configuration (silence thresholds, dropoff/topic node lists, spelling categories, pronunciation phonemes). See references/configuration-guide.md for full payload examples.
6. Validate by running — Execute a small batch and review results. If results look off, check the Common Pitfalls below before reaching for custom metrics.

For full API endpoints (list, toggle, attach, configure, re-evaluate), see references/api-reference.md.

Enabling Predefined Metrics

Two steps are required — missing either means the metric never fires:

1. Toggle on at the project level — enables the metric for simulation runs
2. Add to individual evaluators — attaches the metric to specific test scenarios

Use GET /test_framework/v1/predefined-metrics/ to retrieve the full list of available predefined metrics and their IDs. Pass a predefined metric's code field when adding it to an agent's metric set.

---

Catalog: Accuracy

---

Catalog: Conversation Quality

---

Catalog: Customer Experience

---

Catalog: Speech Quality

---

Configuration Reference

Some predefined metrics require or support configuration. Pass these as key-value pairs in the metric's configuration object.

For full payload examples (including IPA tips and naming guidance) see references/configuration-guide.md.

---

Cost & Credits Quick Reference

---

Baseline — Always Enable

At minimum, every agent should have these four enabled for simulation (and the last three also for observability):

For a richer baseline, also add: CSAT, Sentiment, and Unnecessary Repetition Score.

For agent-type-specific recommended sets (booking, collections, support, healthcare, voice-quality investigation), see references/selection-by-use-case.md.

---

Key Constraints

• Audio required: Transcription Accuracy and Gibberish Detection need audio data. Not available for text-only runs.
• Stereo required (observability): AI Interrupting User, User Interrupting AI, Talk Ratio, and Gibberish Detection require stereo recordings with separate speaker channels for observability calls.
• Simulation only: Transcription Accuracy, Mock Tool Call Accuracy, Expected Outcome.
• Observability only: Dropoff Node, Topic of Call.
• English only: Speaking Rate.
• Requires configuration: Dropoff Node, Topic of Call, Letterwise Pronunciation Detection, Pronunciation Check — will not produce meaningful results without the configuration keys set.
• Requires provider integration: Tool Call Success requires the agent's provider assistant ID configured on Cekura so tool call data appears in transcripts.

---

Common Pitfalls

• Enabling metrics without completing both activation steps (project toggle AND evaluator assignment) — metrics appear available but never fire
• Using Detect Silence and Infrastructure Issues interchangeably — they measure different things (both speakers vs agent only)
• Expecting Transcription Accuracy on observability calls — it's simulation-only
• Forgetting expected_outcome_prompt when using Expected Outcome — without it the metric has nothing to evaluate against
• Using Expected Outcome to evaluate voice characteristics (tone, pronunciation, speech quality) — it only has access to the transcript; use Speech Quality metrics for audio evaluation
• Writing expected_outcome_prompt with terms like "user", "assistant", "bot", or "AI" — always use "main agent" and "testing agent" to match Cekura's transcript labeling
• Enabling Dropoff Node or Topic of Call without configuring dropoff_nodes/topic_nodes — results will be meaningless
• Using Gibberish Detection on mono recordings — it requires stereo audio

---

Next Steps

After selecting predefined metrics, the user typically needs:

• Create or configure metrics → invoke cekura-metric-design for custom metrics to complement predefined ones
• Improve a metric that's underperforming → invoke cekura-metric-improvement for the feedback and labs cycle
• Attach metrics to test scenarios → invoke cekura-eval-design to wire up metrics in evaluators

Documentation

• Pre-defined metrics reference: https://docs.cekura.ai/documentation/key-concepts/metrics/pre-defined-metrics
• Public docs: https://docs.cekura.ai
• Concepts: https://docs.cekura.ai/documentation/key-concepts/

Additional Resources

Reference Files (loaded on demand)

• references/configuration-guide.md — Full payload examples for every configurable predefined metric (silence thresholds, dropoff/topic node lists, spelling categories, IPA phoneme pairs)
• references/api-reference.md — Public API endpoints for listing, toggling, attaching, configuring, and re-evaluating predefined metrics, with an end-to-end example flow
• references/selection-by-use-case.md — Recommended predefined metric sets by agent type: booking, collections, customer support, healthcare, and voice-quality investigation