What is an A/B test?
Showing two versions of a page to different visits and comparing how many reach a goal.
How quietstats counts a/B test
- What it tells you
- Which version of a page did better, and how sure you can be.
- How it is counted
- Visits and goal completions per version, with a two-proportion test.
- Read it well
- Decide in advance what “better” means and how long to run; do not stop early on a lucky day.
- Watch out
- Small tests can’t prove small differences. “No clear winner” is a real answer.
From the metrics handbook: A/B test result.
Worked example
Version A gets 1,000 visits and 30 sign-ups (3.0%); version B gets 1,000 visits and 45 sign-ups (4.5%). B looks better, and a two-proportion test (z ≈ 1.77) says a gap this size would still happen by chance more than 5% of the time, so keep the test running.
Made-up round numbers, to show the arithmetic.
Where to see it
A/B test in quietstats. Exact formulas are on the methodology page.
Try quietstats on your own site
Free for one site and 5,000 pageviews a month, no card needed. No cookies and nothing stored on your visitors’ devices.