Glossary · Validation
Validation harness
External and layered checks that prove behavior; without it, factories produce code that looks correct but is not.
Glossary · Validation
External and layered checks that prove behavior; without it, factories produce code that looks correct but is not.
The validation harness is the set of checks that prove the system meets intent. Practitioners call it the most critical and underbuilt part of many factories.
If validation lives only as in-repo tests, agents can reward-hack: narrow tests can be satisfied trivially (e.g. return true). Strong designs use external scenarios, digital twins of dependencies, tool-based verification (runner, linter, DB, browser), and sometimes LLM-as-judge on natural-language scenarios—with eyes open to judge limitations.
See also: Scenario vs test, Digital Twin Universe (DTU).