CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

testland/mobile-device-matrix-toolkit

Use when deciding which iOS and Android devices and OS versions to test on at each stage to control CI cost: smoke per PR, regression per merge, full farm at release. Generates Appium configs per device from one YAML, runs a GitHub Actions matrix and merges JUnit results into one table.

Quality

84%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

75%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A strong, information-dense skill with executable code, complete CI wiring, an explicit results-validation feedback loop, and a useful anti-patterns table. The main weaknesses are minor: two referenced-but-undefined artifacts (run-tests.sh, browserstack.json), implicit wiring of the reusable run-matrix.yml workflow, hardcoded device/OS version numbers that will age, and inline scripts that imply bundle files that do not exist.

Suggestions

Define or bundle the missing artifacts referenced by CI: show the contents of run-tests.sh (or note that it delegates to the per-platform runner skills) and provide a minimal browserstack.json so the farm wiring is fully copy-paste ready.

Make the workflow wiring explicit: state in Step 2 that the generate-matrix/test job snippet belongs in .github/workflows/run-matrix.yml (the reusable workflow that Step 3 dispatches to), removing the reader's need to infer the connection.

Move the two Python scripts into an actual scripts/ bundle directory and split the per-farm wiring (Firebase/BrowserStack) into a reference file, keeping SKILL.md as a lean overview; this also isolates the time-sensitive device/OS version lists so they can be updated without touching the core skill.

DimensionReasoningScore

Conciseness

The body is dense and table-driven with no library tutorials, but the Overview's combinatorial-explosion framing is partly known to Claude, and the device lists hardcode time-sensitive version numbers ("iPhone 15", "os: 17.4", "api: 34") outside any deprecated/old-patterns section. These are minor instances of over-explanation and time-sensitivity that could be trimmed, matching the 4 anchor rather than 5.

4 / 5

Actionability

Both Python scripts (gen-matrix.py, aggregate-matrix.py) are complete and executable, and the GitHub Actions YAML and gcloud firebase commands are copy-paste ready. Minor gaps keep it below 5: "./scripts/run-tests.sh" is invoked in CI but never defined, and "npx browserstack-runner --config browserstack.json" references a config whose contents are not shown.

4 / 5

Workflow Clarity

A clear Steps 1-5 sequence exists, and Step 4 includes an explicit validation feedback loop for the batch matrix operation ("detect shards with a missing or empty report, re-run only those shards, then re-aggregate"), so the batch-operation cap does not apply. It falls short of 5 because Step 3 calls "./.github/workflows/run-matrix.yml" without ever stating that this is the reusable workflow where Step 2's matrix snippet lives, leaving the wiring implicit.

4 / 5

Progressive Disclosure

Sections are well-organized with clear headers and a closing References list, but the ~280-line body inlines everything: both scripts are labeled as file paths ("# scripts/gen-matrix.py") yet no scripts/ bundle directory exists, and the full device-farm wiring could live in reference files. Good structure with minor organization gaps matches 4; it cannot score 5 under the simple-skill exception since it far exceeds 50 lines and would benefit from real bundle files.

4 / 5

Total

16

/

20

Passed

Description

88%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description that explicitly covers what the skill does and when to use it, with concrete, specific actions and mostly natural trigger terms. Its only weaknesses are a few missing synonyms ("device farm", "emulator/simulator") and slight overlap risk with a plain Appium-testing skill from the "Generates Appium configs" phrasing.

DimensionReasoningScore

Specificity

The description lists multiple specific concrete actions spanning the whole pipeline: "Generates Appium configs per device from one YAML", "runs a GitHub Actions matrix", "merges JUnit results into one table", plus the tiered dispatch decision ("smoke per PR, regression per merge, full farm at release"). This matches the 5 anchor (comprehensive coverage); it is not 4 because the actions cover generation, execution, and reporting with no meaningful gap.

5 / 5

Completeness

It explicitly answers both questions with concrete triggers: the "Use when deciding which iOS and Android devices and OS versions to test on at each stage to control CI cost" clause gives a specific when, and the second sentence gives a clear, concrete what. This matches the 5 anchor; it is not 4 because the when-clause is explicit and stage-specific rather than generic.

5 / 5

Trigger Term Quality

Good natural keyword coverage: "which iOS and Android devices and OS versions to test on", "CI cost", "Appium", "GitHub Actions", "JUnit" - phrases users would plausibly say when needing this skill. A few natural terms are missing, such as "device farm" (only the compressed "full farm" appears), "emulator/simulator", and "mobile testing", which keeps it at 4 rather than 5; it is above 3 because coverage is broad and domain-accurate rather than partially missing.

4 / 5

Distinctiveness Conflict Risk

The skill occupies a clear niche (tiered device-matrix selection for CI cost control) with distinct triggers, but "Generates Appium configs per device" creates minor overlap with a general appium-testing skill - a user asking to write Appium tests could trigger this dispatcher instead. That is minor overlap with a closely related skill, matching 4; it is not 5 due to this overlap and not 3 because the primary trigger (device selection per stage) is clearly distinguishable.

4 / 5

Total

18

/

20

Passed

Validation

93%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 15 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

referenced_paths_exist

Referenced path issues: 3 missing

Warning

Total

15

/

16

Passed

Reviewed

On this page