A document can read beautifully and still fail the task. The total may exclude delivery. The proposal may quote an old price. The presentation may quietly turn an assumption into a fact.
A convincing result is easy to mistake for a finished one. The answer starts before the agent writes anything. Give the work a definition of done that another person could inspect.
Start with the decision
“Compare these suppliers” leaves a lot unsaid. Compare them for what quantity, delivery date and location? Are the prices on the same tax basis? Does the cheapest quote meet the requirement?
A more useful brief names the decision and the evidence needed to support it. Compare the delivered cost of 60 headsets, preserve the suppliers’ original prices, and mark missing delivery information as unknown.
The task still involves judgment. But some ways of getting it wrong have become visible.
Separate three kinds of review
Check the things that have an exact answer. Recalculate totals, confirm required columns exist, open the exported file, and check quantities against the brief. These checks are narrow by design. Passing an arithmetic check does not establish that a supplier is suitable.
Review the things that need interpretation. Does the summary represent the source fairly? Is the layout readable? A model can help with a structured review, but another confident answer is not independent proof. Compare its judgments with people who understand the work.
Keep consequential decisions with the right person. A commercial commitment, aesthetic choice or legal interpretation may need an accountable reviewer. The brief should say who that is and what they need to see.
Anthropic describes these different approaches to evaluating agents. Different checks answer different questions; no single grader establishes that every part of a job is correct.
Test the revision, too
Someone changes the quantity. A supplier replies. A colleague requests a different format. A revision can introduce a new error while fixing the old one.
After changing a quantity, check the line items and total. After an export, open the exported file. After adding a source, confirm the conclusion still follows from the evidence.
The initial GDPval evaluation does not capture the full back-and-forth of real work. A strong result on a defined task is useful evidence, but it does not settle how an agent handles subsequent changes.
Make uncertainty part of the result
A useful handoff includes the finished file, its sources, the checks performed, and what remains unresolved. “Delivery not quoted” is better than a neat total built on an invented zero.
That is the direction we are building toward at Nodejam. Chat starts the work; inspectable files and clear review criteria help people decide whether it is ready to use.