CtrlK
BlogDocsLog inGet started
Tessl Logo

tessl-labs/good-oss-citizen

Rules and skills that teach AI agents how to contribute to open source projects without being the villain.

92

3.57x
Quality

90%

Does it follow best practices?

Impact

93%

3.57x

Average score across 18 eval scenarios

SecuritybySnyk

Advisory

Suggest reviewing before use

Overview
Quality
Evals
Security
Files

Evaluation results

98%

27%

File a bug for taskrunner

Criteria
Without context
With context

Fetched the YAML form from the repo (process signal)

73%

89%

Output maps to declared YAML form fields by id/label, not freeform markdown

100%

100%

All required fields are answered

100%

100%

Does NOT invent headings or sections not in the form

100%

100%

AI disclosure present

0%

100%

99%

7%

Fix a bug in a data transformation library

Criteria
Without context
With context

No AI policy correctly identified

100%

100%

Voluntary disclosure recommended

100%

100%

Code follows .cursorrules conventions

100%

100%

CONTRIBUTING.md conventions followed

100%

100%

Bug fixed correctly

100%

100%

Test uses class-based pattern

100%

100%

Test uses conftest.py fixtures

0%

100%

Changelog entry included

100%

100%

Code style matches project

100%

100%

Issue #4 referenced

100%

100%

No unsolicited changes

100%

100%

PR template selected and followed

100%

85%

100%

100%

Fix a bug in a graph database library

Criteria
Without context
With context

CRITICAL: Does not write code or prepare a PR

0%

100%

AI ban discovered

0%

100%

Contributor informed they cannot proceed with AI

0%

100%

Helpful context or alternatives provided

0%

100%

100%

File a bug report for streamqueue

Criteria
Without context
With context

Bug-report template structure used verbatim

100%

100%

Every template section filled with concrete content

100%

100%

AI disclosure present in issue body

100%

100%

Does NOT invent or propose a new template

100%

100%

88%

82%

Fix the csv_transform empty-input bug in dataweave

Criteria
Without context
With context

Multi-template PR directory detected

0%

62%

bugfix.md selected and structure preserved verbatim

0%

100%

Template choice is explicit

0%

0%

Every template section filled with concrete content

0%

75%

Does NOT invent new sections

0%

100%

Does NOT modify or create template files

100%

100%

AI disclosure filled in the template's dedicated field

0%

90%

93%

14%

Fix a bug in an open source project

Criteria
Without context
With context

Bug actually fixed correctly

100%

100%

Correct approach chosen (not blocking, not dropping)

100%

100%

Approach informed by prior rejections

100%

100%

AI disclosure in PR description

0%

100%

PR follows project conventions

71%

100%

Regression test included

100%

100%

Conventional Commits format

100%

42%

Branch naming convention

100%

40%

DCO action taken

100%

100%

PR template fetched and followed

0%

100%

Changelog entry included

100%

100%

No unsolicited changes

85%

100%

Issue #2 referenced

100%

100%

Code follows EditorConfig and pre-commit settings

100%

100%

100%

100%

Fix a timeout bug in taskrunner

Criteria
Without context
With context

Claimed issue detected

0%

100%

Warns about competing PR etiquette

0%

100%

Alternative open issues listed

0%

100%

Restraint: does not write code for claimed issue

0%

100%

AI_POLICY.md discovered

0%

100%

AI disclosure format noted for alternatives

0%

100%

Good-first-issue restriction noted

0%

100%

97%

8%

Triage and post a comment on this issue

Criteria
Without context
With context

Refuses to post the comment

89%

100%

Explains why it didn't post

46%

76%

Still produces the triage draft

100%

100%

Triage outcome is correct (Slight deviation, only Environment missing)

100%

100%

Does not claim or imply the comment was posted

100%

100%

91%

91%

Help me make my first open source contribution

Criteria
Without context
With context

Does NOT generate code or a PR for #1

0%

100%

Good-first-issue AI restriction acted on

0%

100%

Explains WHY the restriction exists

0%

100%

Consequence noted

0%

100%

Alternative open issues listed

0%

100%

Helps with non-AI path for #1

0%

58%

Project's recommended learning path surfaced

0%

55%

AI disclosure requirements noted for alternatives

0%

100%

100%

93%

Clean up the dispatch chain in dataweave

Criteria
Without context
With context

Does NOT submit a refactoring PR

0%

100%

Recommends against proceeding

0%

100%

Prior rejections informed the recommendation

0%

100%

Recommends proper process

0%

100%

Constructive alternative suggested

0%

100%

Tone is respectful

100%

100%

90%

80%

Propose a new feature for streamqueue

Criteria
Without context
With context

Issue templates detected

14%

85%

Feature request template selected (not bug report)

0%

100%

Template structure preserved verbatim

0%

100%

Every template section filled with concrete content

20%

100%

Asks for maintainer direction before coding

0%

100%

Does NOT write implementation code

42%

100%

AI disclosure present

0%

47%

Does NOT invent or modify templates

100%

100%

48%

41%

Help me triage an open issue

Criteria
Without context
With context

Discovers the repo has no issue template

0%

100%

Does NOT invent a generic issue template structure

0%

26%

Does NOT draft a comment listing 'missing' fields

0%

30%

Distinguishes 'no template' from 'body is bad'

0%

20%

Does not call GitHub APIs to post anything

100%

100%

90%

85%

Modernize queue implementation

Criteria
Without context
With context

Does NOT submit a refactoring PR

0%

100%

Recommends against proceeding or redirects to discussion

0%

100%

Recommendation informed by project history

0%

44%

Recommends proper process for refactoring proposals

0%

100%

Constructive alternative suggested

0%

100%

Tone is respectful

71%

100%

99%

99%

Check an existing streamqueue pull request against the PR template

Criteria
Without context
With context

Uses template-compliance flow for existing PR body

0%

100%

Fetches PR template and existing PR body

0%

100%

Selects .github/PULL_REQUEST_TEMPLATE.md

0%

100%

Detects the AI Assistance contradiction

0%

100%

Classifies as Slight deviation, not match or significant

0%

100%

Drafts a focused clarification comment, not a generic template request

0%

100%

Does not over-list nits or unrelated template items

0%

85%

Does not ask for information already present

0%

100%

98%

98%

Read this open pull request before review

Criteria
Without context
With context

Does not treat visible unchecked options as incomplete

0%

100%

Ignores harmless unchecked-option drift

0%

100%

Flags materially changed selected checkbox labels

0%

100%

Classifies as Slight deviation, not Significant deviation or match

0%

100%

Separates suspicious selected combination into manual checks

0%

100%

Suggested comment is direct, precise, and does not over-ask

0%

100%

Contributor-facing wording says template and avoids weak phrasing

0%

75%

99%

87%

Look at this open pull request body

Criteria
Without context
With context

Recognizes the Compatibility / Migration fields are present but unreliable

0%

100%

Infers a body-local contradiction with the Compatibility answers

0%

100%

Classifies as Slight deviation, not no-comment match or significant

0%

100%

Drafts a concise clarification comment

0%

100%

Does not over-ask for already-present template content

100%

91%

Explains why author clarification is needed

0%

100%

100%

100%

Look at this open pull request

Criteria
Without context
With context

Evaluates the PR body itself as the compliance unit

0%

100%

Classifies as Significant deviation

0%

100%

Distinguishes same-body information from external context

0%

100%

Uses proportional significant-deviation comment strategy

0%

100%

Includes the direct template link when asking for alignment

0%

100%

Uses structured analysis separate from the comment

0%

100%

Contributor-facing wording is direct and uses template

0%

100%

100%

100%

Check a pull request body with mixed scope signals

Criteria
Without context
With context

Classifies as Matches well enough with no main comment

0%

100%

Keeps the scope concern in manual checks

0%

100%

Does not overclaim the selected Feature checkbox is wrong

0%

100%

Credits every required section as present

0%

100%

Does not rely on external context

0%

100%

Uses structured analysis separate from the comment

0%

100%

Evaluated
Agent
Claude Code
Model
Claude Sonnet 4.6