Sphinx extension to support docstrings in Numpy format
52
Pending
Does it follow best practices?
Impact
52%
1.13xAverage score across 9 eval scenarios
Pending
The risk profile of this skill
{
"context": "Evaluates how well the solution uses numpydoc's AST-based linting hook to implement a CLI that respects project configuration discovery, explicit config overrides, and exclusion rules while reporting validation results. Scoring focuses purely on correct integration with numpydoc APIs rather than general coding style.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Config discovery",
"description": "Loads validation settings via numpydoc.hooks.validate_docstrings.parse_config (or equivalent) using project root detection instead of hardcoding defaults, honoring pyproject.toml/setup.cfg precedence.",
"max_score": 25
},
{
"name": "AST lint execution",
"description": "Runs docstring validation through numpydoc's AST workflow (run_hook or process_file/DocstringVisitor) for provided file paths rather than reimplementing checks, propagating exit codes from lint results.",
"max_score": 30
},
{
"name": "Exclude handling",
"description": "Respects exclude patterns from the loaded config when selecting files to lint, leveraging numpydoc's exclusion logic (e.g., find_project_root plus config-driven filtering) so excluded files are skipped cleanly.",
"max_score": 15
},
{
"name": "Config override flag",
"description": "Implements a --config option that passes the supplied path into parse_config (or equivalent loader) to override discovery, ensuring the explicit config fully controls checks/ignore behavior.",
"max_score": 15
},
{
"name": "Error reporting",
"description": "Emits lint findings using numpydoc-provided error details (file, line number, error code/message) and returns exit code 1 when any errors are reported, matching package output semantics.",
"max_score": 15
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9