analyzing-office365-audit-logs-for-compromise
Parse Office 365 Unified Audit Logs via Microsoft Graph API to detect email forwarding rule creation, inbox delegation, suspicious OAuth app grants, and other indicators of account compromise.
61
52%
Does it follow best practices?
Impact
Pending
No eval scenarios have been run
Passed
No known issues
Optimize this skill with Tessl
npx tessl skill review --optimize ./skills/analyzing-office365-audit-logs-for-compromise/SKILL.mdQuality
Discovery
82%Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.
This is a strong, technically specific description that clearly identifies the data source (O365 Unified Audit Logs via Graph API) and concrete detection capabilities (forwarding rules, delegation, OAuth grants, account compromise). Its main weakness is the absence of an explicit 'Use when...' clause, which would help Claude know exactly when to select this skill over others.
Suggestions
Add a 'Use when...' clause such as 'Use when investigating Office 365 account compromise, analyzing audit logs, reviewing suspicious email forwarding rules, or examining OAuth consent grants.'
| Dimension | Reasoning | Score |
|---|---|---|
Specificity | Lists multiple specific concrete actions: 'detect email forwarding rule creation, inbox delegation, suspicious OAuth app grants, and other indicators of account compromise' along with the specific data source 'Office 365 Unified Audit Logs via Microsoft Graph API'. | 3 / 3 |
Completeness | Clearly answers 'what does this do' with specific parsing and detection capabilities, but lacks an explicit 'Use when...' clause or equivalent trigger guidance, which caps this dimension at 2 per the rubric. | 2 / 3 |
Trigger Term Quality | Includes strong natural keywords users would say: 'Office 365', 'Unified Audit Logs', 'Microsoft Graph API', 'email forwarding rule', 'inbox delegation', 'OAuth app grants', 'account compromise'. These are terms a security analyst would naturally use. | 3 / 3 |
Distinctiveness Conflict Risk | Highly distinctive with a clear niche: Office 365 audit log analysis for security incident detection via Microsoft Graph API. Very unlikely to conflict with other skills given the specific domain and technology stack. | 3 / 3 |
Total | 11 / 12 Passed |
Implementation
22%Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.
This skill reads like a high-level outline or table of contents rather than actionable guidance. It correctly identifies the domain and prerequisites but completely lacks executable code, specific API endpoints, query examples, or concrete detection logic. Claude would not be able to perform the described tasks based solely on this content without relying entirely on its pre-existing knowledge.
Suggestions
Add executable Python code for MSAL authentication and Graph API queries, including specific endpoints like `https://graph.microsoft.com/v1.0/users/{id}/mailFolders/inbox/messageRules`
Include concrete example audit log query filters for each suspicious operation (e.g., specific `Operations` values like 'New-InboxRule', 'Set-Mailbox', 'Add-MailboxPermission', 'Consent to application')
Add validation checkpoints: verify authentication succeeded, confirm audit log access, validate returned data schema before processing
Provide a sample JSON output showing the expected compromise indicator report structure with example risk scoring logic
| Dimension | Reasoning | Score |
|---|---|---|
Conciseness | The overview paragraph explains what BEC attacks are and what traces they leave, which is context Claude already knows. The 'When to Use' section is generic boilerplate that adds little value. However, the prerequisites section is useful and specific. | 2 / 3 |
Actionability | The skill provides no executable code, no API endpoints, no concrete commands, no example queries, and no sample payloads. The steps are entirely abstract descriptions of what to do rather than how to do it. There is no copy-paste ready guidance whatsoever. | 1 / 3 |
Workflow Clarity | The 7 steps are listed but are vague descriptions without any validation checkpoints, error handling, or feedback loops. For a multi-step process involving API queries and security analysis, there are no concrete verification steps or decision points for handling failures. | 1 / 3 |
Progressive Disclosure | The content has reasonable section structure (Overview, Prerequisites, Steps, Expected Output) but everything is in one file with no references to detailed materials. The steps section desperately needs expansion or linked reference files for each major operation (authentication, audit log queries, rule enumeration, etc.). | 2 / 3 |
Total | 6 / 12 Passed |
Validation
90%Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.
Validation — 10 / 11 Passed
Validation for skill structure
| Criteria | Description | Result |
|---|---|---|
frontmatter_unknown_keys | Unknown frontmatter key(s) found; consider removing or moving to metadata | Warning |
Total | 10 / 11 Passed | |
- Repository
- mukul975/Anthropic-Cybersecurity-Skills
- Commit
c15f73d
- Reviewed
Table of Contents
Is this your skill?
If you maintain this skill, you can claim it as your own. Once claimed, you can manage eval scenarios, bundle related skills, attach documentation or rules, and ensure cross-agent compatibility.