Async/sync FHIR client for Python providing comprehensive API for CRUD operations over FHIR resources
Agent Success
Agent success rate when using this tile
68%
Improvement
Agent success rate improvement when using this tile compared to baseline
1.08x
Baseline
Agent success rate without this tile
63%
{
"context": "Evaluates how well the solution uses fhir-py's CRUD helpers to manage Patient resources. Checks focus on client setup, creation, retrieval, partial updates, deletion, and returning FHIR-shaped data through library helpers.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Client setup",
"description": "Uses SyncFHIRClient (or AsyncFHIRClient) with the provided base_url and sets the Authorization Bearer header via authorization/extra_headers instead of manual HTTP handling.",
"max_score": 15
},
{
"name": "Patient creation",
"description": "Builds a Patient resource through client.resources('Patient') (or typed model) and persists it using the library's create/save helpers, returning the server-assigned id from the resulting resource.",
"max_score": 20
},
{
"name": "Fetch existing",
"description": "Retrieves patients using the library's get helpers (e.g., search().get or client.resources('Patient').get(id=...)) and maps missing patients to an error using ResourceNotFound rather than custom HTTP calls.",
"max_score": 20
},
{
"name": "Patch contact",
"description": "Performs partial updates of telecom/address via the package's patch or save(fields=[...]) mechanisms so unchanged fields (like names) remain intact, returning the updated resource from the helper call.",
"max_score": 20
},
{
"name": "Delete handling",
"description": "Deletes patients using the library's delete helper on the resource or client; interprets ResourceNotFound as a False result and True when deletion succeeds without hand-rolled HTTP.",
"max_score": 15
},
{
"name": "Resource output",
"description": "Returns patient data using the package's serialize() or resource dict-style accessors to emit FHIR-compliant telecom and address fields instead of manual JSON assembly.",
"max_score": 10
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10