Field notes · November 10, 2026

Does your model notice the change?

Two fictional records show how a model handles a rule that has been replaced.

Does your model notice the change?

The earlier record says 14 days. The updated record says 30 and explicitly supersedes the old rule. A supported answer has to notice the change.

This is one synthetic case in Maine Evidence QA. Its public preview has four records covering a straightforward question, conflicting evidence, missing evidence and an instruction embedded inside a source.

Each example includes its question, source passages, expected answer and rubric. The forestry rule in this case is fictional. It is not a Maine regulation or an operating procedure.

Inspect the whole case.

Look at gen_conflict_0132. Check whether the expected answer follows from the passages, whether the rubric tests the right thing and whether your model distinguishes the old record from the replacement.

The full collection is described as 450 training and 50 validation examples. It is a small experimental collection; no model-quality improvement has been measured. The preview comes from the training file and does not expose the validation records.

Inspect the case and preview

All field notes · Send useful feedback