Understanding Test Results

Every test run in DontBreak produces a detailed report: per-step results with screenshots, a full video recording, console logs, and history stats like success rate and average duration. This page explains how to read a report so you can tell in seconds whether a failure is a real bug, a flaky step, or a change on your site.

Test statuses

In the tests list, each test shows one of four statuses:

Status	Meaning
Draft	Created but not yet run (new tests start here)
Running	A run is currently executing
Passed	The latest run completed with all steps passing
Failed	The latest run had at least one failing step

Each run executes once per enabled browser × device combination (e.g. Chromium Desktop, Firefox Mobile), and the report has a tab for each so you can compare results across browsers and devices.

The report page

The header gives you four tools:

• Logs — the runner's execution log for this report.
• Console — the browser console output, grouped by step, with an error indicator when something logged an error.
• Export — download the report as PDF or JSON.
• Artifacts — download the run's raw artifacts (screenshots, video, traces).

Step-by-step results

The Test Steps panel lists every step with its pass/fail status, duration, and screenshots. Expand a step to see exactly what happened: the screenshot at that moment, the matched element, and — for failures — what was expected versus what the page actually showed.

Each step also has a Jump to this step in video button that seeks the recording to the exact moment the step ran — usually the fastest way to understand a failure.

Video recording

The Test Recording panel plays back the entire run as the browser saw it. Combined with jump-to-step, you can watch the seconds leading up to a failure: was a modal in the way, did the page load slowly, did a banner cover the button?

Match confidence

For steps that target an element, DontBreak finds the element by visual matching — exact pixel matching first, with a feature-detection fallback that's cross-checked against the element's stable DOM identifiers. The step's match details show a confidence percentage — how closely the element found during the run matched the one you captured in the composer — plus the match method used and a thumbnail of the matched area.

High confidence (95%+) means the element looked the same as when you built the test. Lower confidence, or a fallback match method, gets flagged with a warning on the report tab — the step may have still passed, but the element has visually drifted and the step might be worth re-capturing. See Test reliability for keeping confidence high on dynamic pages.

Test history

Below the run details, the Test History section summarizes recent runs:

• Success Rate — percentage of recent runs that passed.
• Avg Duration — average run time, useful for spotting performance regressions.
• Flakiness — how often results flip between pass and fail without changes, a signal of an unreliable step or environment. See Handling dynamic content.
• Runs — passed vs. total runs in the window, with a pass/fail trend chart.

You can export the history as PDF or JSON from this section too.

When a test fails

1. Open the failing step and read the failure detail — assertion failures show what text or element was expected.
2. Jump to this step in video to see the page state at that moment.
3. Check Console for JavaScript errors around that step.
4. Compare browser tabs — a failure in only one browser points to a browser-specific bug.
5. Cross-check the step's match confidence — a low-confidence match may mean the test needs updating, not your site.

For recurring failure patterns, see Common errors. For the full reports list and filtering, see Reports.