Type stubs for chardet - Universal encoding detector for Python
Agent Success
Agent success rate when using this tile
65%
Improvement
Agent success rate improvement when using this tile compared to baseline
0.97x
Baseline
Agent success rate without this tile
67%
{
"context": "Assesses whether the solution builds the stub publish plan by leaning on typeshed's publishing utilities to read METADATA.toml, normalize naming/version data, classify dependencies, and honor default Python version rules. Scoring focuses only on correct use of the package APIs tied to third-party stub publishing.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Metadata reader",
"description": "Loads stubs/<distribution>/METADATA.toml via metadata-aware helpers (e.g., locating files with distribution_path/metadata_path and parsing with read_metadata) instead of manual file parsing.",
"max_score": 25
},
{
"name": "Version & naming",
"description": "Derives stub_distribution and normalized version spec directly from StubMetadata (including the default types-{distribution} fallback) rather than hardcoding values.",
"max_score": 20
},
{
"name": "Upload flags",
"description": "Populates upload_enabled and partial_stub fields from StubMetadata.uploaded_to_pypi/partial_stub so publishability mirrors the metadata toggles.",
"max_score": 15
},
{
"name": "Dependency split",
"description": "Separates internal stub dependencies from external libraries using read_dependencies or get_recursive_requirements so specifiers stay aligned with METADATA.toml intent.",
"max_score": 25
},
{
"name": "Python floor",
"description": "Fills requires_python from StubMetadata.requires_python, falling back to get_oldest_supported_python when unspecified, to match repository defaults.",
"max_score": 15
}
]
}tessl i tessl/pypi-types-chardet@4.0.0docs
evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9