test-oracle-generator
Generates test oracles — the "expected output" part of a test — by choosing among reference implementations, invariants, inverse functions, or differential comparison when the correct answer isn't obvious. Use when the hard part of testing is knowing what the right answer is, not generating inputs.
Install with Tessl CLI
npx tessl i github:santosomar/general-secure-coding-agent-skills --skill test-oracle-generator90
Quality
88%
Does it follow best practices?
Impact
Pending
No eval scenarios have been run
Discovery
85%Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.
This is a strong description that clearly articulates a specific testing niche (the oracle problem) with concrete techniques and explicit usage triggers. The main weakness is trigger term coverage - while technically accurate, it could include more natural language variations that users might employ when struggling with test assertions or expected values.
Suggestions
Add natural trigger terms users might say: 'assertions', 'expected values', 'verify output', 'test expectations', 'what should the test check'
| Dimension | Reasoning | Score |
|---|---|---|
Specificity | Lists multiple specific concrete actions: 'choosing among reference implementations, invariants, inverse functions, or differential comparison' - these are distinct, concrete techniques for generating test oracles. | 3 / 3 |
Completeness | Clearly answers both what ('Generates test oracles by choosing among reference implementations, invariants, inverse functions, or differential comparison') AND when ('Use when the hard part of testing is knowing what the right answer is, not generating inputs'). | 3 / 3 |
Trigger Term Quality | Includes some relevant terms like 'test oracles', 'expected output', 'testing', and 'right answer', but misses common variations users might say like 'assertions', 'test expectations', 'verify results', or 'check output'. | 2 / 3 |
Distinctiveness Conflict Risk | Highly distinctive niche focusing specifically on the oracle problem in testing, with clear differentiation from general test generation skills through the explicit 'when the correct answer isn't obvious' framing. | 3 / 3 |
Total | 11 / 12 Passed |
Implementation
92%Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
This is an excellent skill that efficiently teaches oracle selection with a clear decision tree and concrete, executable examples. The content is dense but well-structured, with the table providing quick reference and worked examples demonstrating each pattern. Minor improvement possible in progressive disclosure by extracting detailed examples to separate files.
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | Every section earns its place. The decision flow, table, and examples are dense with actionable information. No explanations of concepts Claude already knows (like what testing is or how assertions work). | 3 / 3 |
Actionability | Provides fully executable Python code examples with Hypothesis for each oracle type. The worked examples are copy-paste ready with complete implementations of slow_median, invariant checks, and round-trip tests. | 3 / 3 |
Workflow Clarity | The decision flow diagram provides an explicit sequence for choosing oracle types. Each branch has clear criteria and the 'Do not' section provides validation guidance against common mistakes. | 3 / 3 |
Progressive Disclosure | Content is well-organized with clear sections, but everything is inline in one file. The metamorphic relation row references another skill, which is good, but the skill could benefit from separating the worked examples into a reference file for complex use cases. | 2 / 3 |
Total | 11 / 12 Passed |
Validation
100%Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.
Validation — 11 / 11 Passed
Validation for skill structure
No warnings or errors.
- Reviewed
Table of Contents
Is this your skill?
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.