CtrlK
BlogDocsLog inGet started
Tessl Logo
PluginPublic

testland/soc2-evidence-collector

Use when preparing for a SOC 2 Type II audit or any audit that needs evidence packages from automated tests: per-criterion artifacts (CC1-CC9 and optional categories), collection from CI, audit and access logs and change records, Vanta, Drata or Secureframe formats, and observation-period sampling.

Quality

86%

Does it follow best practices?

Run evals on this skill

Adds up to 20 points to the overall score

View guide
SecuritybySnyk

Passed

No findings from the security scan

Overview
Quality
Evals
Security
Files

Quality

Content

78%Weight 40%Scale 1-5

Reviews the quality of instructions and guidance provided to agents. Good implementation is clear, handles edge cases, and produces reliable results.

A well-built workflow skill: lean tables instead of prose, executable examples with an honest account of their injected dependencies, an 8-step sequence validated by a mock-audit step and a worked remediation example, and clean one-level reference structure. The main improvement opportunities are eliminating checklist/reference redundancy and inserting an explicit evidence-validation checkpoint before packaging.

Suggestions

Trim redundancy for conciseness: Step 8's checked list restates Steps 2-7 and the References section re-lists the inline reference links — collapse these into one navigation point.

Make the collector script fully copy-paste ready: create the output directory before writing (`os.makedirs('evidence', exist_ok=True)`) and replace the deprecated `datetime.datetime.utcnow()` with `datetime.datetime.now(datetime.timezone.utc)`.

Add an explicit validation checkpoint between collection and packaging (e.g., after Step 2, verify each evidence JSON is well-formed, non-empty, and covers the full period before wiring it into the GRC platform).

DimensionReasoningScore

Conciseness

The body is dense and table-driven with no beginner-concept padding, and the genuinely non-obvious notes (injected client wrappers, 25-40 sample sizing, continuity-gap findings) earn their tokens. It falls short of anchor 5 due to redundancy: Step 8's checked list restates Steps 2-7 and the References section re-lists links already given inline. Anchor 3 does not fit — there is no unnecessary concept explanation.

4 / 5

Actionability

The collector script is complete and runnable, the CC6.3 test is concrete, and the wrapper interfaces are explicitly explained as injected clients. Minor gaps keep it below anchor 5: the script writes to an `evidence/` directory it never creates, `datetime.utcnow()` is deprecated, and the test snippet depends on undefined `User`/`deprovision` helpers — clearly above anchor 3's pseudocode level.

4 / 5

Workflow Clarity

Steps 1-8 are clearly sequenced, with a Step 8 checklist, a mock-auditor validation step ("request a sample; verify response is complete + timely"), and a worked example showing a full fail→fix→green feedback loop. Not anchor 5 because validation is concentrated at the end rather than as explicit checkpoints between steps — e.g., no stated verify-evidence-before-packaging step; not anchor 3 because validation is present, so the batch-operation cap does not apply.

4 / 5

Progressive Disclosure

Clear overview with three well-signaled, one-level-deep references, all of which exist in the bundle ("The full control-to-evidence-source table is in references/evidence-source-map.md", etc.), and the two deep-dive references cross-link only to each other rather than nesting further. Detailed tables and the packaging/CI workflows are appropriately split out of SKILL.md, matching the anchor-5 structure.

5 / 5

Total

17

/

20

Passed

Description

87%Weight 40%Scale 1-5

Based on the skill's description, can an agent find and select it at the right time? Clear, specific descriptions lead to better discovery.

A strong description: explicit "Use when" triggers, concrete deliverables and source systems, and platform names that make it highly distinctive. Its only weakness is that capabilities are expressed as a dense noun list rather than crisp action verbs, which slightly blurs the "what it does" reading.

DimensionReasoningScore

Specificity

Names concrete deliverables and sources — "per-criterion artifacts (CC1-CC9 and optional categories)", "collection from CI, audit and access logs and change records", "observation-period sampling" — but the only action verb is "preparing"; capabilities are stated as noun phrases rather than actions. It sits above anchor 3 (multiple concrete items named) but below anchor 5, whose examples lead with comprehensive action verbs.

4 / 5

Completeness

"Use when preparing for a SOC 2 Type II audit or any audit that needs evidence packages from automated tests" gives an explicit, concrete trigger, and the colon-delimited list explicitly states what is produced (per-criterion artifacts, log/change-record collection, GRC formats, sampling). Anchor 4 does not fit because the "when" clause is already fully explicit and specific, not something that "could be more explicit".

5 / 5

Trigger Term Quality

Good natural-keyword coverage: "SOC 2 Type II audit", "evidence packages", "Vanta, Drata or Secureframe", "CI", "audit". Common variations a user might say — "compliance", "readiness", "Trust Services Criteria", "controls" — are missing, so it falls short of anchor 5's comprehensive synonym coverage but is clearly above anchor 3.

4 / 5

Distinctiveness Conflict Risk

Occupies a clear niche — automated SOC 2 evidence collection — with highly specific triggers (Type II, CC1-CC9, Vanta/Drata/Secureframe, observation-period sampling) that no generic audit or document skill would match. Minimal conflict risk with adjacent compliance skills since the platform and criterion names disambiguate it.

5 / 5

Total

18

/

20

Passed

Validation

100%

Checks the skill against the spec for correct structure and formatting. All validation checks must pass before discovery and implementation can be scored.

Validation — 16 / 16 Passed

Validation for skill structure

No warnings or errors.

Reviewed

On this page