or run

tessl search
Log in

Version

Workspace
tessl
Visibility
Public
Created
Last updated
Describes
pypipkg:pypi/chex@0.1.x
tile.json

tessl/pypi-chex

tessl install tessl/pypi-chex@0.1.0

Comprehensive utilities library for JAX testing, debugging, and instrumentation

Agent Success

Agent success rate when using this tile

73%

Improvement

Agent success rate improvement when using this tile compared to baseline

1.92x

Baseline

Agent success rate without this tile

38%

rubric.jsonevals/scenario-4/

{
  "context": "This criteria evaluates how well the engineer uses chex's tree comparison and path-based error reporting capabilities to implement the nested structure validation tool. The focus is on utilizing chex functions that provide detailed diagnostic information about mismatches in nested data structures.",
  "type": "weighted_checklist",
  "checklist": [
    {
      "name": "Uses tree comparison",
      "description": "Uses chex tree comparison functions such as assert_trees_all_equal() or assert_trees_all_equal_comparator() to compare the nested structures",
      "max_score": 30
    },
    {
      "name": "Leverages path reporting",
      "description": "Leverages chex's built-in path-based error reporting which automatically includes the path to mismatched elements in error messages when tree assertions fail",
      "max_score": 25
    },
    {
      "name": "Handles structure types",
      "description": "Properly handles comparison of different structure types (dictionaries, lists, tuples) using chex's PyTree understanding, without manually implementing tree traversal",
      "max_score": 20
    },
    {
      "name": "Type checking",
      "description": "Uses chex functions like assert_trees_all_equal_dtypes() or assert_trees_all_equal_shapes_and_dtypes() to detect type mismatches, or relies on assert_trees_all_equal() which checks types",
      "max_score": 15
    },
    {
      "name": "Error propagation",
      "description": "Allows chex's AssertionError to propagate naturally without catching and re-raising with custom messages, preserving the detailed path information that chex provides",
      "max_score": 10
    }
  ]
}