Python docstring style checker for PEP 257 compliance
72
{
"context": "Evaluates whether the solution leverages pydocstyle to lint docstrings while correctly honoring or disabling inline # noqa suppression toggles. Checks are limited to how pydocstyle is invoked and how its Error objects are mapped into the returned violations.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Uses pydocstyle",
"description": "Lint execution is driven by pydocstyle APIs such as pydocstyle.check(...) or ConventionChecker.check_source(...) instead of reimplementing docstring parsing, and results come from the package's Error objects.",
"max_score": 30
},
{
"name": "Inline honored",
"description": "When respect_inline_noqa is True, the call keeps ignore_inline_noqa=False (or equivalent configuration) so inline # noqa comments suppress D1xx and other eligible codes as pydocstyle intends.",
"max_score": 25
},
{
"name": "Toggle disables",
"description": "When respect_inline_noqa is False, the code passes ignore_inline_noqa=True into pydocstyle.check/ConventionChecker so previously suppressed violations reappear.",
"max_score": 20
},
{
"name": "Code-specific",
"description": "Inline selectors like # noqa: D400 rely on pydocstyle's built-in inline parser so only the listed codes are suppressed and other codes in the same file are still reported.",
"max_score": 15
},
{
"name": "Error mapping",
"description": "Returned Violation entries are populated directly from pydocstyle Error fields (code, filename/path, line, message text) and keep deterministic ordering by path then line.",
"max_score": 10
}
]
}Install with Tessl CLI
npx tessl i tessl/pypi-pep257evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10