Async/sync FHIR client for Python providing comprehensive API for CRUD operations over FHIR resources
Agent Success
Agent success rate when using this tile
68%
Improvement
Agent success rate improvement when using this tile compared to baseline
1.08x
Baseline
Agent success rate without this tile
63%
{
"context": "Evaluates whether the solution relies on fhirpy's SearchSet query builder to compose and run Observation/Patient queries for the specified tasks. Checks correct use of search filters, limits, sorting, element selection, includes, and reverse chaining instead of manual HTTP calls or client-side filtering.",
"type": "weighted_checklist",
"checklist": [
{
"name": "Search filters",
"description": "Uses SearchSet.search (or SQ) parameters to apply patient reference, code list, and since filters rather than constructing URLs manually or filtering results after fetch.",
"max_score": 20
},
{
"name": "Limit & sort",
"description": "Applies SearchSet.limit(max_results) and SearchSet.sort('-effectiveDateTime' or equivalent recency field) so returned Observations/Patients respect size caps and newest-first ordering.",
"max_score": 20
},
{
"name": "Element selection",
"description": "Uses SearchSet.elements(...) (with exclude flag when needed) to restrict Observation fields for summary_only responses instead of trimming data manually.",
"max_score": 15
},
{
"name": "Includes",
"description": "Adds include clauses via SearchSet.include (and revinclude if used) to bundle related Patient and Practitioner resources with Observation queries, avoiding separate fetches or manual joins.",
"max_score": 15
},
{
"name": "Reverse chaining",
"description": "Finds Patients with matching Observations through SearchSet.has(...) or explicit _has parameters (via SQ), not by fetching Observations then filtering/iterating in Python.",
"max_score": 20
},
{
"name": "Search execution",
"description": "Executes queries with SearchSet fetch helpers (fetch, fetch_all, or fetch_raw) and uses include_observations paths to return bundles rather than hand-built HTTP requests.",
"max_score": 10
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
scenario-6
scenario-7
scenario-8
scenario-9
scenario-10