Katalon Scout
An autonomous web testing agent that turns a URL into a bug report.
Katalon Scout
Live
Scout is running in production for real teams — the agent browses on its own and returns structured findings, shipped on the same trunk as Katalon's own engineers from first vision-based proof-of-concept through to live product.
4
Reporting lenses every run covers — performance, usability, accessibility, and console — scoped by intent, so a single URL turns into a reproducible quality report instead of a screenshot dump.
Katalon builds quality tooling used by teams that ship serious software — from individual testers up through large engineering organisations. Scout is Katalon’s autonomous web testing agent: paste a URL, and the agent browses the site on its own, returning a structured report of what’s broken, slow, or confusing.
We put a small embedded team alongside Katalon’s — deeply in the AI engineering loop, owning the agent runtime, the browser-automation plumbing, and the product surface engineers and testers actually meet. The remit ran from first vision-based browse to live product: shipping on the same trunk as Katalon’s own engineers, with feature flags carrying the riskier behaviours.
Scout is in production today, serving real teams. The agent runs on a Mastra-orchestrated stack with Playwright handling the browser and Bedrock handling the model calls, deployed across three Railway services. The dual-path architecture keeps the safer fallback one flag away.
Quality teams burn hours clicking through websites looking for things that broke. Katalon wanted a faster path: paste a URL, get a structured bug report — performance, usability, accessibility, console — without anyone holding the mouse. The product had to feel less like a script and more like a tester you can hand a link to. We embedded with Katalon's team and built Scout from the first vision-based browsing proof-of-concept through to a live product, shipping on the same trunk as Katalon's own engineers.
Scout takes screenshots as primary input, uses vision to read page state, and types into forms the way a human would. The interesting work was the unglamorous part. Multi-metric screenshot hashing kept the agent from re-processing the same view a dozen times and burning tokens for nothing. A focus guard stopped it from clicking away from an input mid-type — a subtle bug that quietly broke every form-heavy run before we hunted it down. Page-load and network-idle timeouts were tuned against real SPAs, with a 100KB snapshot ceiling so the agent's working memory stayed reasonable on heavy pages.
Scout decomposes a job through a planner, then dispatches sub-agents to execute individual steps. Early on, the sub-agents were strangers to the plan — they knew what to click but not why. We threaded the parent plan into every sub-agent's context so each step is taken with the whole task in view, which was the change that finally made multi-step runs hold together. The deep-bug hook layered on top of that: when Scout finds something worth investigating, it can pivot from surface observation into a real follow-up — clicking into the issue, varying inputs, mapping the boundaries — instead of just noting it and moving on.
Not every run wants the same depth. Quick Bug is a one-step sprint for fast smoke checks. Smart is the default — five steps, balanced for everyday investigation. Verification goes deeper for accessibility audits and end-to-end coverage. Each intent has its own step budget, agent behaviour, and report shape, and the router picks early so the run feels tailored from the first click. The Roast feature adds an even lighter on-ramp: a quick quality glance from a URL, with an embeddable badge that lives on a site like a CI status pill — a low-friction way for teams to feel the value before committing to a full execution.
Scout runs on a Mastra-orchestrated agent backed by Playwright for browser control and Bedrock for model calls, deployed to Railway as three services — the admin web app, the agent itself, and the user-facing front end. Thanh built the surface engineers actually meet: Markdown reports, a Live View tab that follows the run, suggestion buttons in place of free-text prompts, and a Storybook component library for the chain-of-thought UI, todo dots, and shimmer effects that became the agent's visual language. The whole stack ships behind feature flags so new behaviours can be turned on for one organisation at a time, with the safer path one toggle away.
A few sentences about your team and what you're building is usually enough. We reply within three working days.