captcha-detection-metrics

Interpreting and querying Firefox's `captcha_detection.*` Glean metrics and the `captcha-detection` custom ping (BigQuery `mozdata.firefox_desktop.captcha_detection`, `mozdata.fenix.captcha_detection`). Use when analyzing captcha prevalence or solve / pass / interacted rates per vendor (ArkoseLabs, Cloudflare Turnstile, Datadome, reCAPTCHA v2, hCaptcha, AWS WAF), cohorting the ping by privacy settings or browsing volume, or writing or reviewing a query or dashboard over it. Records which ratios are valid per vendor, which counters are dead or over-counting, and the filters that keep non-organic rows out.

Quality

81%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

Securityby

Passed

No findings from the security scan

Quality

Content

63%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A dense, genuinely non-obvious analysis guide with concrete formulas, SQL filters, and well-sequenced validation logic, but it is verbose and monolithic — repeated caveats and time-sensitive details are inlined rather than isolated, and the long per-vendor reference is not split into bundle files.

Suggestions

Move the per-vendor metric references (ArkoseLabs, Cloudflare Turnstile, Datadome, reCAPTCHA v2, hCaptcha) into separate files under references/ and keep SKILL.md as an overview that links to them, reducing inline length and improving progressive disclosure.

Consolidate the reCAPTCHA `ps` over-counting explanation (currently repeated in the reCAPTCHA, hCaptcha, and Invalid-ratios sections) into one location and cross-reference it.

Gather time-sensitive facts (Firefox 154 cutoff, bug 2054037/2054272/2054072 statuses, the 2026-07-13 figures) into a dedicated 'Known metric gaps / time-sensitive' section so the stable guidance reads cleaner and staleness is easy to spot.

Dimension	Reasoning	Score
Conciseness	Mostly high-value non-obvious domain knowledge, but the ~480-line body is noticeably verbose: the reCAPTCHA `ps` over-counting is restated across the reCAPTCHA, hCaptcha, and Invalid-ratios sections, and time-sensitive facts (Firefox 154, 'As of 2026-07-13', bug numbers) are woven through main sections rather than isolated in a deprecated/old-patterns block.	3 / 5
Actionability	Provides concrete ratio formulas and executable SQL snippets ('EXTRACT(DAYOFWEEK FROM DATE(submission_timestamp)) BETWEEN 2 AND 6', 'NOT is_bot_generated', 'LHS > 2*RHS + 10') plus real table/column paths, but stops short of a complete end-to-end copy-paste query.	4 / 5
Workflow Clarity	The Data Analysis Techniques section sequences cohorts → time filtering → anomaly exclusion → querying with explicit validation checkpoints (per-ping invalid-ratio drop, converging load-spike rule with iteration) and a feedback loop, with only minor gaps.	4 / 5
Progressive Disclosure	Section headers are well organized, but with no bundle files in references/scripts/assets the entire ~480-line per-vendor reference is inlined in SKILL.md; content that would naturally live in separate reference files (per-vendor metric specs, cohort definitions) is not split out.	3 / 5
	Total	14 / 20 Passed

Description

100%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is exemplary: third-person gerund voice, explicit 'what' and 'Use when' triggers, comprehensive concrete actions, and a distinct niche with rich natural trigger terms including all vendor synonyms. No fluff or over-claims.

Dimension	Reasoning	Score
Specificity	Names multiple concrete actions — 'Interpreting and querying', 'analyzing captcha prevalence or solve/pass/interacted rates per vendor', 'cohorting the ping by privacy settings or browsing volume', 'writing or reviewing a query or dashboard' — with comprehensive coverage of the skill's scope.	5 / 5
Completeness	Explicitly states what it does ('Interpreting and querying Firefox's captcha_detection.* Glean metrics…') and when to use it ('Use when analyzing captcha prevalence… or writing or reviewing a query or dashboard over it') with concrete trigger phrases.	5 / 5
Trigger Term Quality	Covers natural phrases users would say — 'captcha prevalence', 'solve/pass/interacted rates', 'query or dashboard', 'privacy settings', 'browsing volume' — plus the full vendor list (ArkoseLabs, Cloudflare Turnstile, Datadome, reCAPTCHA v2, hCaptcha, AWS WAF) as synonyms/triggers.	5 / 5
Distinctiveness Conflict Risk	Targets a narrow, well-defined niche — Firefox's captcha_detection Glean ping in BigQuery — with vendor and metric names that make conflict with other skills minimal.	5 / 5
	Total	20 / 20 Passed

Validation

100%

Warnings & errors only

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository: mozilla/firefox-aidev-plugins
Path: plugins/captchadetection/skills/captcha-detection-metrics/SKILL.md
Commit: 6821231

Reviewed: 1 day ago

Table of Contents

Discovery Implementation Validation

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.