Test planner: how long should an A/B test run?
Most tests on small sites stop too early and crown a winner by chance.
How many visits and days an A/B test needs at your traffic to find a 5%, 10%, 20%, 30% or 50% lift.
Why quietstats: you get this without cookies, a consent banner or storing IP addresses, in every plan including Free.
In every plan, including Free. Goals, events and conversions.
📘 Step-by-step guide, with screenshots and diagrams →
Start free, no cardSee live demo
- No cookies or consent banner
- No IP addresses stored
- Groups under 5 visitors hidden
- 845-byte script
- Free plan, no card


What it is
Using your current conversion rate for the goal and your visits per day, quietstats works out the visits needed for a two-version test (50/50 split, 95% confidence, 80% power) for each lift, and how many days that takes.
Why it matters
Most tests on small sites stop too early and crown a winner by chance. Planning the length first avoids that.
How it works
- Open your dashboard, go to Deep dives (or Reports on a phone) and open Analysis lab.
- Under Tools, tap Test planner. You can also search for it on the All reports screen.
- Pick a date range (and any options at the top), read the “How to read this” notes, and download any table as CSV.
Privacy: what is and isn’t collected
Collected
- Nothing new: it reads the pageviews, events, goals or daily totals you already have
Never collected
- Anything about a single visitor
- Links between days: visitor ids still reset at midnight
- Cookies, localStorage or anything else stored on the visitor’s device
- IP addresses or user agents (used in memory, then discarded)
- Anything that links a visitor across days (the visitor ID resets daily)
Rows and limits
- Needs at least 20 visits in the range
- Assumes a 50/50 split and a steady rate
- Up to 92 days and 200,000 pageviews per screen
- Read from the database only when you open it, then kept for 10 minutes
“Rows” are database rows written, which is what our hosting is billed on. Reads are not rows written. Methodology →
Questions
What if it says over a year?
Test bigger changes (they have bigger lifts), or test on a page with more traffic.
Does it store anything new?
No. It is worked out from the pageviews and events you already have when you open the screen, and only the totals are kept for 10 minutes.
Related features
Goals, events and conversionsGoals and funnels
Goals, events and conversionsMargin of error: which gaps are real?
Data in and outWhat-if calculator: where is the biggest win?
Goals, events and conversionsEvents by landing page: which first pages lead to action
Goals, events and conversionsPage targets: monthly visitor goals for single pages
Goals, events and conversionsGoal overlap: which goals happen together
Goals, events and conversionsCampaign planner: plan campaigns, then track them against the plan
Sources and campaigns
All goals, events and conversions features → · All features → · Docs: Test planner guide → · Compare features with other tools →
Guides on this topic
Set it up
Nothing to set up: it works with the standard one-line script.
<script data-host="https://quietstats.co.uk" data-dnt="true" src="https://quietstats.co.uk/js/script.js" id="YOUR_SITE_ID" async defer></script>Try it on your own site
Free during launch: one site and 5,000 pageviews a month, no card needed. After launch, paid plans start at £5 a month.