Async/sync FHIR client for Python providing comprehensive API for CRUD operations over FHIR resources
Agent Success
Agent success rate when using this tile
68%
Improvement
Agent success rate improvement when using this tile compared to baseline
1.08x
Baseline
Agent success rate without this tile
63%
{
"context": "Evaluates how well the solution relies on fhir-py’s nested path helpers to read and mutate FHIR resource dictionaries using dot/index paths. Scoring checks for direct use of the package’s getter/setter utilities (and their defaults) rather than manual traversal. Emphasis is on correct helper calls that match the paths and behaviors described in the spec.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Path retrieval",
"description": "Uses fhir-py path readers (e.g., resource.get_by_path or fhirpy.base.utils.get_by_path) to resolve each dotted path in read_paths, rather than manually walking dictionaries/lists.",
"max_score": 30
},
{
"name": "Default handling",
"description": "Passes the default argument supported by the path reader so missing paths return the provided fallback instead of custom None checks or try/except logic.",
"max_score": 15
},
{
"name": "Path setting",
"description": "Uses fhir-py’s setter (fhirpy.base.utils.set_by_path) to write values into nested structures for updates/creations, instead of hand-building intermediate dicts/lists.",
"max_score": 30
},
{
"name": "Copy via helpers",
"description": "Implements copy_path by combining the package’s get_by_path and set_by_path (or resource helpers) to move values between paths without custom traversal code.",
"max_score": 15
},
{
"name": "Path syntax fidelity",
"description": "Helper calls use fhir-py’s expected dot-and-index path strings (e.g., \"telecom.0.value\", \"address.0.city\") without bespoke path parsing logic.",
"max_score": 10
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10