CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

sentinel-solution-mock-data-generation

Generate or reuse one shared mock.json payload for a Microsoft Sentinel Analytic Rule and its converted Defender XDR Custom Detection. Uses only Azure-Sentinel repository content and never writes to Azure.

Quality

73%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Fix and improve this skill with Tessl

tessl review fix ./.github/skills/sentinel-solution-mock-data-generation/SKILL.md
SKILL.md
Quality
Evals
Security

Quality

Content

71%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

This is a highly actionable, safety-conscious skill: exact commands, exact contracts, and explicit validation/review gates make it hard to execute incorrectly. Its weaknesses are repetition (six near-identical CLI invocations, duplicated output-layout blocks) and the absence of progressive disclosure — rule-specific Cisco ASA detail and full status documentation are inlined rather than offloaded to reference files.

Suggestions

Collapse the three near-identical Cisco ASA invocations into one template plus a short per-rule table (rule ID, detection filename, candidate JSON), and merge the duplicated output-layout blocks into a single section.

Move the per-rule Cisco ASA predicate requirements and the full status/manifest vocabulary into a references/ file (e.g. references/cisco-asa-rules.md and references/status-contract.md) with clearly signaled one-level-deep links from SKILL.md.

Add an explicit error-recovery loop for the documented failure states — e.g. "if the scenario is blocked or schema validation fails, report the blocking field/table to the user and do not retry with guessed mappings" — to upgrade workflow clarity.

DimensionReasoningScore

Conciseness

The body is information-dense and assumes domain competence (no generic KQL/Sentinel tutorials), but it repeats the same `generate_mock_scenario.py` invocation six times with near-identical parameter blocks (including three consecutive Cisco ASA examples that differ only in rule IDs and filenames), and the output-layout block appears twice ("Existing-scenario-first flow" and "Output contract"). This fits anchor 3 — mostly efficient but could be tightened by collapsing the repeated command blocks — rather than anchor 2, since the padding is repetition of genuinely operational content, not explanation of concepts Claude already knows.

3 / 5

Actionability

Every flow ships copy-paste-ready PowerShell commands with exact flags (`--solution`, `--rule`, `--custom-detection`, `--candidate-scenario`, `--seed`, `--lock`, `--benign-set`), exact input paths, a precise JSON contract with per-field rules ("`name` (non-empty string), `hypothesis` (non-empty string), optional `lockedPaths`..."), concrete rule IDs and detection filenames for the Cisco ASA cases, and enumerated output/status values. This matches anchor 5: fully executable commands covering the common cases.

5 / 5

Workflow Clarity

The sequence is clear and gated: check for an existing reviewed scenario first, run with reuse semantics (`status=existing-scenario`), the 9-step automatic-generation list ends in field-level validation, statuses like `generated-offline-only` and `review-pending` act as explicit checkpoints, and the numbered Handoff section closes the process. It falls short of anchor 5 because there is no explicit fix-and-retry loop for the failure states it documents (blocked scenarios, validation failures) — the tool blocks, but the skill never instructs what to do next — leaving most-but-not-all checkpoints covered, which is anchor 4.

4 / 5

Progressive Disclosure

There are no bundle files at all (no references/, scripts/, or assets/), so everything lives in a ~280-line SKILL.md. Section headers are clear, but rule-specific material that clearly belongs in a separate reference — the Cisco ASA per-rule predicate requirements, the three example invocations, and the full candidate-manifest status vocabulary — is inlined in the main file. This matches anchor 3 (some structure, content that should be separate is inline) rather than anchor 4, which expects most content appropriately split across files.

3 / 5

Total

15

/

20

Passed

Description

65%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

The description is highly specific to its Sentinel/XDR niche and nearly conflict-free, with an explicit safety boundary ("never writes to Azure"). Its main weaknesses are the absence of an explicit 'Use when...' trigger clause, which caps completeness, and a narrow action inventory that undersells the skill's validation and reuse behavior.

Suggestions

Append an explicit trigger clause, e.g. "Use when generating or reusing mock data (mock.json) for a Sentinel Analytic Rule or its converted Defender XDR Custom Detection, or when preparing test fixtures for migration/optional-testing workflows."

Include natural synonyms users would say — "mock data", "test data", "fixture", "Sample Data" — alongside "mock.json" to broaden trigger coverage.

Mention one or two of the concrete capabilities currently only in the body (schema validation of every field, existing-scenario reuse with review) so the action list is more comprehensive.

DimensionReasoningScore

Specificity

"Generate or reuse one shared mock.json payload for a Microsoft Sentinel Analytic Rule and its converted Defender XDR Custom Detection" names the domain precisely and gives two concrete actions (generate, reuse a shared payload), but coverage is narrow — no mention of validation, scenario review, or the reuse/refresh flow that the body delivers. This matches anchor 3 (domain plus 1-2 concrete actions, not comprehensive) rather than anchor 4, which requires several listed actions.

3 / 5

Completeness

The 'what' is clear and specific (generate/reuse a shared mock.json payload for an AR and its converted CD, using only repo content, never writing to Azure), but there is no 'Use when...' clause or equivalent explicit trigger guidance; per the judging guidelines a missing trigger clause caps completeness at 3. It is not a 2 because the 'what' is concrete, not vague.

3 / 5

Trigger Term Quality

Strong natural domain keywords: "Microsoft Sentinel Analytic Rule", "Defender XDR Custom Detection", "mock.json", "Azure-Sentinel repository" — exactly the phrases a user doing Sentinel migration work would say. A few common variants are missing ("mock data", "test data", "fixture", "sample data", "migration"), keeping it just below the comprehensive-synonym level of anchor 5 and clearly above the sparse keyword coverage of anchor 3.

4 / 5

Distinctiveness Conflict Risk

"Microsoft Sentinel Analytic Rule", "converted Defender XDR Custom Detection", and "Azure-Sentinel repository" define a clear niche with distinct triggers and essentially no overlap risk with generic data-generation or testing skills, matching anchor 5.

5 / 5

Total

15

/

20

Passed

Validation

87%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 14 / 16 Passed

Validation for skill structure

CriteriaDescriptionResult

allowed_tools_field

'allowed-tools' contains unusual tool name(s)

Warning

frontmatter_unknown_keys

Unknown frontmatter key(s) found; consider removing or moving to metadata

Warning

Total

14

/

16

Passed

Repository

Azure/Azure-Sentinel

Path

.github/skills/sentinel-solution-mock-data-generation/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.