Auto-syncs stale docstrings and README when function signatures change. Detects documentation drift after refactors, parameter additions, or return type changes. Dry-run by default — proposes before writing.
87
100%
Does it follow best practices?
Impact
86%
1.59xAverage score across 17 eval scenarios
Passed
No known issues
A platform engineering team tightened the validation logic inside a Python function. The function's signature is completely unchanged — same parameters, same return type — but the implementation now enforces an additional rule. The existing docstring describes the old behavior and is now inaccurate.
The team wants to catch any stale documentation before shipping. Use --apply mode.
Extract the following files, then set up the git repository:
=============== FILE: inputs/setup.sh =============== #!/usr/bin/env bash set -euo pipefail git init git config user.email "dev@example.com" git config user.name "Dev" mkdir -p src cat > src/validator.py << 'PYEOF' def validate_password(password: str) -> bool: """Validate a password against security policy.
Checks two conditions:
1. Minimum length of 8 characters
2. Contains at least one digit
Args:
password: The plaintext password string to validate.
Returns:
True if valid, False otherwise.
"""
if len(password) < 8:
return False
if not any(c.isdigit() for c in password):
return False
return TruePYEOF git add -A && git commit -m "baseline" cat > src/validator.py << 'PYEOF' def validate_password(password: str) -> bool: """Validate a password against security policy.
Checks two conditions:
1. Minimum length of 8 characters
2. Contains at least one digit
Args:
password: The plaintext password string to validate.
Returns:
True if valid, False otherwise.
"""
if len(password) < 8:
return False
if not any(c.isdigit() for c in password):
return False
if not any(c.isupper() for c in password):
return False
if not any(c in "!@#$%^&*" for c in password):
return False
return TruePYEOF
Sync the documentation and write the results to doc-sync-report.md.
evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10
scenario-11
scenario-12
scenario-13
scenario-14
scenario-15
scenario-16
scenario-17