Methodology
Rules we hold ourselves to
- We only publish results we ran. No "we tested" without a run, no scores borrowed from vendors presented as ours. Tools we haven't run are labelled not yet benchmarked.
- Everything is re-runnable. Each round publishes its task definitions, success checks, runner script and per-run raw results.
- Versions and environment are pinned and printed on the results page: library versions, browser build, machine, network, date.
- Success is machine-checked. Every task has an explicit check (a count, an exact value, a visible element). No subjective grading.
- Facts carry sources and dates. Repository data comes from the GitHub REST API with the fetch time shown; descriptive claims link to the project's own README or docs.
- Ranking is never for sale. If we join affiliate programmes later, order and verdicts still come only from the data, and affiliate links will be marked and carry
rel="sponsored". See the disclosure. - Mistakes are corrected in public on the corrections page.
Timing
Wall-clock time measured in the runner process with performance.now(), from after browser launch until the success check returns. Browser launch is measured separately. We report medians and min–max over successful runs rather than means, because network-bound timings have long tails.
Target sites
We use sites that are published for automation practice (for example toscrape.com sandboxes and the-internet.herokuapp.com) or that we host ourselves. We keep request volume low and never target sites whose terms forbid automated access.