Async/sync FHIR client for Python providing comprehensive API for CRUD operations over FHIR resources
Agent Success
Agent success rate when using this tile
68%
Improvement
Agent success rate improvement when using this tile compared to baseline
1.08x
Baseline
Agent success rate without this tile
63%
{
"context": "Evaluates whether the solution builds and executes the Observation search using fhir-py query utilities. Checks correct use of chaining, comparison formatting, reference conversion, boolean encoding, and raw parameter passthrough.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Chained identifier",
"description": "Patient constraint is built with fhir-py's chained search support (e.g., SQ(subject__Patient__identifier=...) or SearchSet.search with double-underscore chaining) instead of manual query strings.",
"max_score": 25
},
{
"name": "Datetime bound",
"description": "Uses fhir-py's comparison helpers (such as SQ(effective__ge=recorded_after) or search(date__ge=recorded_after)) to format the lower-bound datetime with the proper ge prefix rather than hand-crafting the timestamp string.",
"max_score": 20
},
{
"name": "Reference formatting",
"description": "Formats performer filtering through fhir-py's reference helpers (e.g., client.reference('Practitioner', performer_id) or to_reference()) so the query sends a Practitioner/{id} reference instead of concatenated strings.",
"max_score": 20
},
{
"name": "Boolean handling",
"description": "Passes boolean status flags directly into SearchSet.search so fhir-py performs the true/false encoding, avoiding manual string literals for the status filter.",
"max_score": 15
},
{
"name": "Raw passthrough",
"description": "Adds any extra expression via fhirpy.base.searchset.Raw (or equivalent Raw wrapper) to append the literal query segment without transformation.",
"max_score": 20
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10