CtrlK
CommunityDocumentationLog inGet started
Tessl Logo

tessl/pypi-types-chardet

Type stubs for chardet - Universal encoding detector for Python

Agent Success

Agent success rate when using this tile

65%

Improvement

Agent success rate improvement when using this tile compared to baseline

0.97x

Baseline

Agent success rate without this tile

67%

Overview
Eval results
Files

rubric.jsonevals/scenario-4/

{
  "context": "Evaluates whether the stubtest planning utility leverages ts_utils to gather platform-specific stubtest requirements and allowlists instead of reimplementing metadata parsing or path logic. It checks reliance on the provided helpers for path discovery, platform handling, and argument construction.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Stubtest settings",
      "description": "Uses ts_utils.metadata.read_stubtest_settings to load per-distribution settings and honors skip flags and ci_platforms when deciding whether to include a distribution.",
      "max_score": 20
    },
    {
      "name": "Platform deps",
      "description": "Retrieves system requirements via StubtestSettings.system_requirements_for_platform or ts_utils.requirements.get_stubtest_system_requirements (with platform override when provided) instead of hardcoding package names, and de-duplicates/sorts the result.",
      "max_score": 20
    },
    {
      "name": "Allowlist set",
      "description": "Determines which allowlist files to consider using ts_utils.utils.allowlists so stdlib and third-party distributions follow their prescribed filename patterns rather than custom heuristics.",
      "max_score": 20
    },
    {
      "name": "Allowlist args",
      "description": "Builds the stubtest --allowlist arguments using ts_utils.utils.allowlist_stubtest_arguments (or equivalent logic backed by allowlists_path) and omits missing files instead of assuming they exist.",
      "max_score": 15
    },
    {
      "name": "Path helpers",
      "description": "Resolves directories for tests/allowlists via ts_utils.paths helpers (tests_path/allowlists_path) or their outputs rather than assembling paths manually.",
      "max_score": 10
    },
    {
      "name": "Plan assembly",
      "description": "Combines requirement and allowlist data into the final plan by orchestrating ts_utils calls (not by duplicating rules), preserving deterministic ordering for both sections.",
      "max_score": 15
    }
  ]
}
tessl i tessl/pypi-types-chardet@4.0.0

tile.json