When a user submits a teaching interaction, a front-end agent creates a benchmark that PAT should fail, confirming that the poor behavior can be reproduced, then iterates on context/harness until the benchmark passes, and finally confirms no regression across the broader test suite before submitting a pull request.

factualpending

Speaker

Michael Ran

Evidence Quote

“an agent on the back end will create a benchmark that we expect to fail. This shows that we can reproduce this poor behavior. It will then iterate on our context repositories or the harness itself until that benchmark passes. And then after confirming that by making this benchmark pass, we didn't cause the rest of our suite to fail, we get a Slack message with the pull request, including the changes the agent wants to make to PAT.”

Source

How Bridgewater Built an AI Analyst That Does Hours of Expert Research in Minutes— LangChain
Created: 8/12/2026, 6:15:31 PM

Your notes

Loading notes...