or run

tessl search
Log in

Version

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/atheris@2.3.x
tile.json

tessl/pypi-atheris

tessl install tessl/pypi-atheris@2.3.0

A coverage-guided fuzzer for Python and Python extensions based on libFuzzer

Agent Success

Agent success rate when using this tile

91%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.28x

Baseline

Agent success rate without this tile

71%

rubric.jsonevals/scenario-10/

{
  "context": "This criteria evaluates how well the engineer uses Atheris's data flow tracing capabilities to build a fuzzer that discovers magic numbers and secret strings through comparison tracking. The focus is on proper instrumentation setup and leveraging _trace_cmp functionality.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Fuzzer initialization",
      "description": "Correctly uses atheris.Setup() to initialize the fuzzer with sys.argv and a test callback function",
      "max_score": 15
    },
    {
      "name": "TestOneInput callback",
      "description": "Implements a proper TestOneInput callback that accepts bytes and passes them to the validation function",
      "max_score": 10
    },
    {
      "name": "FuzzedDataProvider usage",
      "description": "Uses atheris.FuzzedDataProvider to consume structured data (integers and strings) from raw fuzzing input",
      "max_score": 15
    },
    {
      "name": "Function instrumentation",
      "description": "Applies instrumentation to the validation function using atheris.instrument_func() decorator or atheris.instrument_imports() to enable data flow tracing",
      "max_score": 25
    },
    {
      "name": "Comparison operations",
      "description": "Validation function contains comparison operations (== for integers and strings) that will be traced by _trace_cmp when instrumented",
      "max_score": 15
    },
    {
      "name": "Fuzzing execution",
      "description": "Calls atheris.Fuzz() to start the fuzzing loop and discover inputs that satisfy the validation checks",
      "max_score": 10
    },
    {
      "name": "Integer comparison tracing",
      "description": "Implements integer comparison operations (== with magic number 0x41424344) that leverage _trace_cmp for data flow tracking",
      "max_score": 5
    },
    {
      "name": "String comparison tracing",
      "description": "Implements string comparison operations (== with 'SECRET_KEY') that leverage _trace_cmp for discovering the correct secret value",
      "max_score": 5
    }
  ]
}