New Study: LLM Judges Score Patent Drafting Unevenly Against Human Attorneys

It is an end-to-end patent-drafting testbed for AI agents, where a separately invoked LLM judge evaluates generated drafts and provides structured feedback for iterative revision.

The study finds real but bounded utility: agreement with a professional patent attorney exists but varies by metric, with systematic calibration differences, meaning the judge is a useful optimization signal rather than a final authority.

It can be piloted as an assistive layer to speed first drafts and cut cost, but certified human review should remain mandatory, especially where local legal standards differ from training data.