A Journey to the East for Quality

When you let an AI agent write your tests, it looks confident — and gets the big things wrong. Run it over your codebase and it will produce a tidy "test" that verifies nothing: red marked green, a bug declared a feature, an invalid result swallowed as correct.

We lived through this. We turned a loose collection of scripts into a system of agents with strict role separation, and along the way had to answer nine uncomfortable questions — exactly the ones this talk addresses. What counts as a bug and what counts as a feature? How do you handle legacy services with no documentation? What actually happens when you swap the model underneath?

The talk is a map of our experience, not a textbook: real problems and concrete fixes.