triage-suspicious-login
Triage suspicious login alerts like impossible travel, untrusted location, or multiple failures. Use when investigating authentication anomalies. Analyzes user history, source IP reputation, login patterns, and determines if escalation is needed.
88
86%
Does it follow best practices?
Impact
Pending
No eval scenarios have been run
Passed
No known issues
Quality
Discovery
100%Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.
This is a strong skill description that clearly defines its security-focused domain with specific capabilities (triaging login alerts, analyzing IP reputation, determining escalation needs). It includes an explicit 'Use when' clause with natural trigger terms and provides enough detail to distinguish it from other skills. The description uses proper third-person voice throughout.
| Dimension | Reasoning | Score |
|---|---|---|
Specificity | Lists multiple specific concrete actions: 'Triage suspicious login alerts', 'Analyzes user history, source IP reputation, login patterns', and 'determines if escalation is needed'. Also specifies alert types: 'impossible travel, untrusted location, or multiple failures'. | 3 / 3 |
Completeness | Clearly answers both what ('Triage suspicious login alerts... Analyzes user history, source IP reputation, login patterns, determines if escalation is needed') AND when ('Use when investigating authentication anomalies') with explicit trigger guidance. | 3 / 3 |
Trigger Term Quality | Includes natural keywords users would say: 'suspicious login', 'impossible travel', 'untrusted location', 'multiple failures', 'authentication anomalies', 'IP reputation', 'login patterns'. Good coverage of security/SOC terminology. | 3 / 3 |
Distinctiveness Conflict Risk | Clear niche in security/authentication domain with distinct triggers like 'suspicious login', 'impossible travel', 'authentication anomalies'. Unlikely to conflict with general document or code skills. | 3 / 3 |
Total | 12 / 12 Passed |
Implementation
72%Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
This is a well-structured SOC triage skill with clear workflow steps, useful decision matrices, and good pattern documentation. The main weaknesses are inconsistent command syntax (mixing full tool calls with shorthand references like `/enrich-ioc`) and missing validation/error handling steps in the workflow. The required outputs table is a strong addition for accountability.
Suggestions
Standardize tool call syntax - either show full MCP tool calls for all steps or document the shorthand notation (e.g., show what `/enrich-ioc` expands to)
Add validation checkpoints after critical steps, e.g., 'If lookup_entity returns no results, check for typos in USER_ID or search for aliases'
Fix the search query example to show proper variable interpolation instead of literal 'USER_ID' string
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The skill is lean and efficient, avoiding explanations of concepts Claude already knows. Each section serves a clear purpose with no padding or unnecessary context about what suspicious logins are or how SOC operations work. | 3 / 3 |
Actionability | Provides concrete tool calls and commands, but some are pseudocode-like (e.g., `/enrich-ioc` and `/document-in-case` without full syntax). The search query uses placeholder 'USER_ID' in quotes rather than showing proper variable substitution. | 2 / 3 |
Workflow Clarity | Steps are clearly sequenced and numbered, but lacks explicit validation checkpoints. No feedback loops for error recovery if tool calls fail or return unexpected results. The decision matrix helps but doesn't integrate into the workflow as verification steps. | 2 / 3 |
Progressive Disclosure | Well-organized with clear sections: workflow steps, required outputs table, decision matrix, and key patterns. Content is appropriately structured for a single skill file without needing external references for this scope. | 3 / 3 |
Total | 10 / 12 Passed |
Validation
90%Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.
Validation — 10 / 11 Passed
Validation for skill structure
| Criteria | Description | Result |
|---|---|---|
frontmatter_unknown_keys | Unknown frontmatter key(s) found; consider removing or moving to metadata | Warning |
Total | 10 / 11 Passed | |
- Repository
- dandye/ai-runbooks
- Commit
4d132c7
- Reviewed
Table of Contents
Is this your skill?
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.