Async/sync FHIR client for Python providing comprehensive API for CRUD operations over FHIR resources
Agent Success
Agent success rate when using this tile
68%
Improvement
Agent success rate improvement when using this tile compared to baseline
1.08x
Baseline
Agent success rate without this tile
63%
{
"context": "Evaluates how well the solution uses fhir-py to build paired sync and async clients with matching configuration. Focuses on using the library's search and resource helpers for patient retrieval and observation creation across both execution modes.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Sync setup",
"description": "Creates SyncFHIRClient with the provided base URL, passes the token to the authorization parameter, applies extra_headers, and forwards client_config into requests_config instead of crafting custom HTTP calls.",
"max_score": 20
},
{
"name": "Async setup",
"description": "Creates AsyncFHIRClient with the same base URL and headers, forwarding client_config into aiohttp_config, and uses the async client directly rather than wrapping the sync path.",
"max_score": 20
},
{
"name": "Patient fetch",
"description": "Uses client.resources('Patient') search helpers (search/get/first/fetch) to retrieve the patient record and derive id/name/birthDate from the returned resource or serialize(), avoiding manual HTTP or hardcoded data.",
"max_score": 20
},
{
"name": "Observation create",
"description": "Builds an Observation via client.resource('Observation', ...) or resources('Observation') with subject=client.reference('Patient', id=patient_id), code, and valueQuantity, then persists it with create/save/patch to obtain the server-assigned id.",
"max_score": 25
},
{
"name": "Async parity",
"description": "Async methods mirror the sync logic with AsyncFHIRClient operations (resources/search/create/fetch) and await the calls, not by invoking sync methods through executors or blocking adapters, and return the expected id/field shapes.",
"max_score": 15
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10