I wanted to know what AI could actually do in software testing — so I built the system, ran the experiments, investigated the failures, and documented what I learned.
{{ item.text }}
This book will not tell you AI has solved software testing. It will tell you, precisely, what it did and didn't do when actually built and run.
A test suite of thirty-four scenarios had passed for months, covering every corner of a demand-planning module built and maintained from scratch. Then, over the course of a week, the pass rate dropped sharply — with no corresponding change to the scenarios themselves.
The traditional response: open the CI report, read the failure logs one scenario at a time, reproduce locally, inspect the DOM in DevTools to find out whether a locator changed or the application regressed, fix or file a defect, rerun. One particular week was lost to a bug that required no judgment at all — a filename casing mismatch between a local machine and a case-sensitive CI runner.
"What if the testing system could investigate its own failures?"
Not "AI writes my tests for me" — Codegen and a Page Object Model already did that. Something further: a system that sees a failure, gathers the same evidence a human would gather, forms a hypothesis, proposes or applies a fix, and reports — honestly — whether it worked. That question became HeliosQE.
The system takes a plain-English requirement and attempts a full pipeline — plan, generate, execute, investigate, repair, report — with a real stack (React, Node/Express, Prisma, Postgres) and real model calls (Anthropic and Gemini APIs). If a module wasn't backed by a real model call against real data, it was never called "live."
Four narrow agents, each individually inspectable, testable, and replaceable — instead of one opaque loop doing every judgment call at once.
Click a stage to see its interface.
An earlier version used a single agent with every tool available. It worked — and was impossible to debug safely, because one prompt handled planning, code generation, and failure diagnosis at once. Splitting responsibilities made each decision individually inspectable.
{{ b.text }}
Fourteen parts, forty chapters, moving from the problem to a working architecture to what it's actually worth in production.
"A green test is not automatically proof that the AI fixed the right thing."
A repair loop's job is to make a test pass. Nothing in that objective guarantees the resulting test still verifies what it was originally meant to verify. Self-healing works best on narrow, mechanical changes — a relabeled button, a renamed CSS class. It fails when it papers over an ambiguous element, a misunderstood business rule, or a genuine application defect.
Vague claims ("it works most of the time") aren't good enough. The book defines the metrics it actually measured its own architecture against.
Five documented experiments is not a statistically rigorous benchmark, and this book doesn't pretend otherwise by inventing precision it doesn't have. What the experiments support, directionally: classification (Reviewer) performed more reliably than generative repair (Developer), and grounding locators in real browser observation reduced — without eliminating — invented-selector failures. Reliability must be measured empirically, not asserted.
Tool access, browser control, and execution capability are an expansion of attack surface — not a neutral engineering decision.
Never let a system whose job is "make the test pass" have unsupervised authority over questions of "is the application actually correct."
Every LLM call costs money and takes time — and repair cycles multiply both. Before deploying an agentic workflow, understand what every loop and model call costs.
Five pages, straight from the manuscript.
{{ f.a }}
AI testing is changing quickly. Learn how to reason about it before blindly adopting it.
You've seen the architecture. You've seen the experiments. You've seen what worked, and what failed.