or run

tessl search
Log in

Version

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/atheris@2.3.x
tile.json

tessl/pypi-atheris

tessl install tessl/pypi-atheris@2.3.0

A coverage-guided fuzzer for Python and Python extensions based on libFuzzer

Agent Success

Agent success rate when using this tile

91%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.28x

Baseline

Agent success rate without this tile

71%

rubric.jsonevals/scenario-1/

{
  "context": "This criteria evaluates the engineer's understanding and correct usage of atheris counter management APIs, specifically focusing on the UpdateCounterArrays() function for registering coverage counters after dynamic instrumentation.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Uses _reserve_counter()",
      "description": "Code calls atheris._reserve_counter() to allocate unique counter IDs for coverage tracking. Should be called at least twice to demonstrate multiple counter allocation.",
      "max_score": 20
    },
    {
      "name": "Calls UpdateCounterArrays()",
      "description": "Code calls atheris.UpdateCounterArrays() to register allocated counters with libFuzzer. Must be called after reserving counters and after instrumenting code.",
      "max_score": 30
    },
    {
      "name": "Uses instrument_func()",
      "description": "Code uses atheris.instrument_func() decorator to instrument a dynamically created function for coverage tracking.",
      "max_score": 20
    },
    {
      "name": "Correct sequencing",
      "description": "Demonstrates the correct workflow: reserve counters before or during instrumentation, then call UpdateCounterArrays() after instrumentation to register the counters.",
      "max_score": 15
    },
    {
      "name": "Instrumentation verification",
      "description": "Code verifies that instrumentation was applied by checking for '__ATHERIS_INSTRUMENTED__' marker in the code object's constants (co_consts).",
      "max_score": 10
    },
    {
      "name": "Multiple registrations",
      "description": "Code demonstrates that UpdateCounterArrays() can be called multiple times as new counters are allocated, showing understanding that registration can happen incrementally.",
      "max_score": 5
    }
  ]
}