Error handling for ASP.NET Core APIs — exception middleware, ProblemDetails,
94
90%
Does it follow best practices?
Impact
100%
1.13xAverage score across 5 eval scenarios
Passed
No known issues
{
"context": "Tests whether the agent configures InvalidModelStateResponseFactory to return ValidationProblemDetails with the correct title, status, instance, and Content-Type, ensuring model validation errors use the same RFC 7807 format as other errors.",
"type": "weighted_checklist",
"checklist": [
{
"name": "InvalidModelStateResponseFactory configured",
"description": "Program.cs calls .ConfigureApiBehaviorOptions(options => { options.InvalidModelStateResponseFactory = ...; }) to override the default validation response",
"max_score": 18
},
{
"name": "Returns ValidationProblemDetails",
"description": "The factory returns a ValidationProblemDetails object (not a plain ProblemDetails or anonymous object), so the 'errors' dictionary is populated from ModelState",
"max_score": 14
},
{
"name": "Title set to VALIDATION_ERROR",
"description": "The ValidationProblemDetails Title is set to 'VALIDATION_ERROR' (exact string)",
"max_score": 12
},
{
"name": "Status set to 400",
"description": "The ValidationProblemDetails Status is explicitly set to 400 (or StatusCodes.Status400BadRequest)",
"max_score": 10
},
{
"name": "Instance set to request path",
"description": "The ValidationProblemDetails Instance is set to context.HttpContext.Request.Path",
"max_score": 10
},
{
"name": "Content-Type application/problem+json",
"description": "The BadRequestObjectResult (or equivalent return) sets ContentTypes to include 'application/problem+json'",
"max_score": 14
},
{
"name": "At least four validation attributes",
"description": "PatientRegistrationRequest.cs has at least four data annotation validation attributes across its properties (e.g. [Required], [MaxLength], [RegularExpression], [Range], [EmailAddress])",
"max_score": 8
},
{
"name": "No manual validation in controller",
"description": "PatientsController.cs does NOT contain manual if-checks for model field values — validation is handled by the framework (no if (string.IsNullOrEmpty(request.Name)) etc.)",
"max_score": 8
},
{
"name": "validation_design.md mentions errors dictionary",
"description": "validation_design.md mentions the 'errors' field/dictionary in the response and notes that it contains field-level error messages",
"max_score": 6
}
]
}evals
scenario-1
scenario-2
scenario-3
scenario-4
scenario-5
skills
aspnet-error-handling
verifiers