CtrlK
BlogDocsLog inGet started
Tessl Logo
SkillPublic

update-solution-analyzer

MANDATORY guidelines for ANY edit to ANY file under Tools/Solutions Analyzer/ — including the mapper (map_solutions_connectors_tables.py), doc generator (generate_connector_docs.py), interactive docs (generate_interactive_docs.py), ASIM browser, collect_table_info, collect_asim_fields, upload_to_kusto, compare_runs, solution_analyzer_overrides.csv, filter_field_resolution.yaml, or any other script/config in that folder. Use when: editing/modifying/refactoring/fixing/renaming/tweaking ANY logic in those files, even one-line fixes such as renaming a classification, escaping a character, adjusting a regex, adding an override row, suppressing a false positive, or changing a constant. Covers: keeping script-docs in sync, README Version History changelog rules (required for feature/behavior changes; optional for small bug fixes), CSV output sync with upload_to_kusto.py, static/interactive index synchronization, and markdown/HTML entity page synchronization.

Quality

91%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

SKILL.md
Quality
Evals
Security

Quality

Content

83%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A lean, well-structured conventions skill with genuinely actionable, project-specific rules (exact files, lists, and section names to touch for each change type). Its main gap is the absence of validation/verification checkpoints: the workflow tells Claude what to update but never how to confirm the synchronized artifacts are actually in sync after an edit.

Suggestions

Add a validation step to the workflow, e.g., 'After changes, re-run generate_connector_docs.py and generate_interactive_docs.py and diff the regenerated index/entity pages to confirm static and interactive outputs are in sync.'

Provide an ordered post-edit checklist (docs → changelog → upload_to_kusto → both index sets → HTML pages) so the multi-artifact update sequence is explicit rather than implied by section order.

Include one concrete example of a well-formed Version History changelog entry (heading, bold category, bullet) so the required format is unambiguous.

DimensionReasoningScore

Conciseness

Every section conveys project-specific rules Claude could not know (changelog version-entry rules, dual index synchronization, markdown-to-HTML derivation) with no explanation of known concepts and no padding. This matches the level-5 anchor ('lean and efficient; assumes Claude's competence; every token earns its place'); it is clearly above level 4, where unnecessary explanation would remain.

5 / 5

Actionability

Guidance is concrete and executable throughout: exact paths ('script-docs/csv/<csv-name>.md', 'Tools/Solutions Analyzer/README.md'), a named code target ('upload_to_kusto.py → SOLUTION_ANALYZER_FILES list'), named index files, and specific sub-steps ('add the new CSV to script-docs/csv/README.md (both the "By generating script" and "By role" sections)'). Per the code-vs-instruction scoring note, absence of code is not penalized for an instruction-only skill, and this guidance is fully actionable — matching the level-5 anchor.

5 / 5

Workflow Clarity

The tasks are grouped into clear sections and a 'Before Modifying a Script' step exists, but there is no explicit end-to-end sequence and, critically, no validation checkpoints — nothing like 're-run generate_connector_docs.py to regenerate and verify the indexes/pages stay in sync' after making changes. For a skill whose whole purpose is keeping generated, batch-produced artifacts synchronized, the missing verify/regenerate feedback loop caps this at the level-3 anchor ('sequence present but checkpoints missing or implicit').

3 / 5

Progressive Disclosure

The ~65-line body is well-organized into six clearly headed concern areas with no nested references and no wall of text, matching the level-4 anchor ('good structure; most content is appropriately placed; references mostly clear'). It falls short of 5 only because it slightly exceeds the under-50-lines simple-skill threshold and the changelog rules or index-sync bullet lists could arguably live in a one-level-deep reference file.

4 / 5

Total

17

/

20

Passed

Description

95%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A highly effective trigger description: it explicitly defines both what it covers and when to use it, with unusually thorough synonym and edge-case trigger coverage ('even one-line fixes such as...'). Its only real weakness is verbosity — the padding and repeated enumeration could be trimmed without losing any triggering power.

DimensionReasoningScore

Specificity

Names the domain precisely (Tools/Solutions Analyzer/ with specific scripts like map_solutions_connectors_tables.py, upload_to_kusto) and lists multiple concrete coverage areas ('keeping script-docs in sync, README Version History changelog rules... CSV output sync with upload_to_kusto.py, static/interactive index synchronization, and markdown/HTML entity page synchronization'), approaching the level-5 anchor. However, the ~160-word length with padded emphasis ('MANDATORY', 'ANY edit to ANY file', repeated enumerations) is padded relative to the concise-and-clear ideal, so it lands at 4 rather than 5.

4 / 5

Completeness

Explicitly answers both questions: 'what' via the 'Covers:' clause enumerating the five guideline areas, and 'when' via the explicit 'Use when: editing/modifying/refactoring/fixing/renaming/tweaking ANY logic in those files, even one-line fixes' clause with concrete trigger phrases. This is a direct match for the level-5 anchor; it is clearly not level 4, where the 'when' would be less specific.

5 / 5

Trigger Term Quality

Comprehensive natural trigger coverage with synonyms ('editing/modifying/refactoring/fixing/renaming/tweaking'), file extensions embedded in named files (.py, .csv, .yaml), and concrete user-shaped scenarios ('renaming a classification, escaping a character, adjusting a regex, adding an override row, suppressing a false positive'). This matches the level-5 anchor's 'comprehensive coverage of natural terms including synonyms and file extensions'; only the word 'update' itself is oddly absent from the verb list, which is too minor to drop to 4.

5 / 5

Distinctiveness Conflict Risk

Scoped to a single named folder and specific scripts/configs (ASIM browser, solution_analyzer_overrides.csv, filter_field_resolution.yaml), giving it a clear niche with distinct triggers and minimal conflict risk with other skills. It does not resemble the overlapping broad-domain examples of lower anchors.

5 / 5

Total

19

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Repository

Azure/Azure-Sentinel

Path

.github/skills/update-solution-analyzer/SKILL.md

Reviewed

Table of Contents

Is this your skill?

If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.