What it requires. A prompt template or action asked for a set of things - every case, each product, all the reasons - with no structured output and no completeness check, so a response missing items reads exactly like a complete one.
Why it matters. An evaluator that returns a schema keyed by rule code cannot omit a rule without failing validation. With no such guard, coverage drift is invisible: rules go unevaluated behind a plausible-looking total, and nothing distinguishes a run that checked everything from one that quietly checked less. With it, a run either carries the expected rows for every subject - 1,147 checks of 1,147 - or it fails outright. A generation with no output schema has no equivalent guard, and a short answer is a well-formed one.
How to fix it. Give the template an output schema listing what must come back, so an incomplete answer fails instead of reading as complete.
Source. Our evaluator coverage design, August 2026
Is this rule worth checking?
Votes and comments are published here. We read them, and we publish what we change with the reasoning — a vote does not move a rule on its own.