CtrlK
BlogDocsLog inGet started
Tessl Logo

litert-model-equivalence-test

Validates equivalence between LiteRT models (litert_lm) and PyTorch models (transformers). Use when you need to verify that an exported LiteRT model produces the same outputs as the original Hugging Face model. Supports multi-turn conversations and custom prompts.

68

Quality

81%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide

SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

72%

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A clean, highly actionable single-file skill with executable commands and well-organized sections. The two gaps are conciseness (the long bazel target is repeated verbatim in every example) and workflow clarity (no description of expected pass/fail output or failure-handling guidance for a validation skill).

Suggestions

Factor the shared `bazel run //third_party/py/litert_torch/generative/export_hf/experimental/validation:equivalence_test --` target out of the examples (define it once, then show only the per-example flags) to remove the repeated multi-line invocation and tighten token cost.

Add a short "Interpreting results" note describing what a passing run prints versus a mismatch/failure, and one or two recovery steps (e.g., re-run with `--max_new_tokens` lowered, or check that export flags match the source config) to give the validation workflow an explicit checkpoint and feedback loop.

Replace the opening restatement ("This skill provides instructions for running equivalence tests...") with a one-line pointer to the Usage section, since the frontmatter description already covers it.

DimensionReasoningScore

Conciseness

The body is lean and assumes Claude's competence (no explanations of what LiteRT, bazel, or KV caches are), but the full multi-line `bazel run //third_party/py/litert_torch/.../equivalence_test --` invocation is repeated verbatim across all five examples plus the usage block, and the opening line restates the frontmatter description; per the guideline to penalize verbosity even when accurate, this could be tightened by factoring out the shared target, so it sits at score 2 rather than 3.

2 / 3

Actionability

It provides fully executable `bazel run` commands with real flag values and defaults, a per-flag reference, and five concrete copy-paste scenarios (single-turn, multi-turn, prompt file, externalized embedder, NPU variant), matching the score-3 "fully executable code/commands; specific examples; copy-paste ready" anchor; it is not score 2 because nothing is pseudocode or missing key details.

3 / 3

Workflow Clarity

The single action (run the equivalence test) is unambiguous and well-sequenced, but for a skill whose entire purpose is validation the body never states what a passing vs. failing result looks like, nor what to do on failure — an implicit validation-checkpoint gap that fits the score-2 anchor ("sequence present but checkpoints missing or implicit") rather than the score-3 anchor requiring explicit validation steps and error-recovery feedback loops.

2 / 3

Progressive Disclosure

It is a single, well-organized SKILL.md with clearly labeled sections (Usage, Running the Test, Flags, Examples) and easy navigation, no nested references, and no need for external bundle files; per the simple-skills scoring note this earns a 3 with well-organized sections even without external references, and it is not score 2 because nothing that should be split out is inlined in a disorganized way.

3 / 3

Total

10

/

12

Passed

Description

90%

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong, third-person description with an explicit "Use when" trigger, natural domain keywords, and a distinct niche. Its only weakness is specificity: it conveys one core action with feature elaboration rather than a list of multiple distinct concrete actions.

Suggestions

Broaden the action list to name multiple distinct concrete capabilities (e.g., "compare token-level outputs, diff logits, and report mismatched turns") so specificity reaches the multi-action bar of the top anchor.

Drop the parenthetical package names ("(litert_lm)", "(transformers)") from the description if they are not terms users naturally say, keeping only the trigger keywords that aid discovery.

DimensionReasoningScore

Specificity

It names the domain and a concrete primary action ("Validates equivalence between LiteRT models ... and PyTorch models", "verify that an exported LiteRT model produces the same outputs") plus supporting features ("multi-turn conversations and custom prompts"), but it does not enumerate multiple distinct concrete actions the way the score-3 anchor ("Extract text and tables ... fill forms, merge documents") does; it is a single core action with feature elaboration rather than a comprehensive action list.

2 / 3

Completeness

It explicitly answers both halves: "what" ("Validates equivalence between LiteRT models ... and PyTorch models") and "when" via an explicit "Use when you need to verify that an exported LiteRT model produces the same outputs" trigger, which is the exact form of the score-3 anchor; it is not the score-2 case because the trigger is stated explicitly rather than merely implied.

3 / 3

Trigger Term Quality

It surfaces the natural terms a user in this domain would actually say — "LiteRT models", "PyTorch models", "Hugging Face model", "equivalence", "exported LiteRT model", "multi-turn conversations", "custom prompts" — with good coverage and no jargon-for-its-own-sake, matching the score-3 anchor for natural keyword coverage.

3 / 3

Distinctiveness Conflict Risk

The niche is sharply bounded — LiteRT-vs-PyTorch/transformers model equivalence testing — with triggers unlikely to fire for unrelated skills, matching the score-3 "clear niche with distinct triggers" anchor rather than the overlapping score-2 case.

3 / 3

Total

11

/

12

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository
google-ai-edge/litert-torch
Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.