unit test
In software, an automated test that checks whether a specific piece of code produces the correct output; here used as an example of measurement data for training models.
As defined in Brendan Foody
Where it comes up · 2
Most important claims · 2 of 2Search all claims for “unit test” →
- There are two kinds of data: output/curriculum data the model reads and learns from, and measurement data (rubrics, test questions and answers, unit tests) that lets models attempt a problem many times, score responses, and learn from them—and the second kind is the most valuable.Brendan Foody
- When an LM's code fails a unit test, prompting the LM to explain why and suggesting changes enables it to generate improved versions (version 3) that work better, creating a feedback loop.What's next for AI agentic workflows ft. Andrew Ng of AI Fund