Start free

How to use test planner: how long should an A/B test run?

How many visits and days an A/B test needs at your traffic to find a 5%, 10%, 20%, 30% or 50% lift.

Goals & funnelsFeature page: Test planner: how long should an A/B test run?
Test planner: how long should an A/B test run?: example screenshotTest planner: how long should an A/B test run?: example screenshot
Example screenshot with made-up data, not a real customer’s numbers.

What it does

Using your current conversion rate for the goal and your visits per day, quietstats works out the visits needed for a two-version test (50/50 split, 95% confidence, 80% power) for each lift, and how many days that takes.

Why it matters

Most tests on small sites stop too early and crown a winner by chance. Planning the length first avoids that.

How to use it

  1. Open Deep dives → Analysis lab → Test planner, or search “Test planner” on All reports (or go to /app/lab/testplanner?site=YOUR_SITE).
  2. Pick a goal at the top, then a lift.
  3. The default range is 30 days; pick another at the top.
  4. Download any table with its CSV link; small numbers show as “<5” there too.

How it works

How Test planner is worked out1. Visits and conversions for the goal are counted. 2. The current rate and visits per day are worked out. (otherwise: Only totals are kept, never single visits.) 3. The sample size for each lift is worked out with the standard formula. 4. Days needed at your traffic are shown.1Visits and conversions for thegoal are counted.2The current rate and visitsper day are worked out.Only totals arekept, neversingle visits.3The sample size for each liftis worked out with thestandard formula.4Days needed at your trafficare shown.
From src/lab2.js (testplanner) and src/labdata.js.

Privacy

Limits

Plans

Every plan, including Free. quietstats is free during launch: the Free plan covers 1 site and 5,000 pageviews a month, and no card is needed. See pricing.

Questions

What if it says over a year?

Test bigger changes (they have bigger lifts), or test on a page with more traffic.

Does it store anything new?

No. It is worked out from the pageviews and events you already have when you open the screen, and only the totals are kept for 10 minutes.

Related

Feature page: Test planner: how long should an A/B test run? → What it is, why it matters, privacy and how other tools do it.

Guides on this topic

All website analytics guides →

Related features

Browse all 448 features →

Explore quietstats

See your traffic (119)

Live visitors and live eventsTrend charts, headline cards and any date rangeCompare any two date rangesWorld map, countries and regionsDevices, browsers, operating systems and screens

Sources and campaigns (63)

Sources and channelsAI search referralsUTM campaignsSearch keywords from Google Search ConsoleShort links

Behaviour and content (80)

Top pagesPage transitionsScroll depth and engaged timeClick mapVisitor journeys

Goals, events and conversions (83)

Goals and funnelsCustom events and conversion ratesRevenue cards and revenue by sourceEngagement goalsA/B tests with your own tags and a plain-English confidence note

Reports, sharing and alerts (44)

Alerts inbox and rulesReport snapshotsPassword-protected snapshotsOptional public stats pageAnonymous benchmarks against similar sites

Data in and out (42)

CSV and Excel downloadsStats APIImport from Plausible or GA4API playgroundTSV and XML export for every report

Privacy and performance (17)

Zero cookies, privacy built inUnder 1 KB script and single-page appsPages never countedPrivacy proof page and badge for your siteSigned-in sessions

Why quietstats

Why quietstatsPricingLive demoStart free

Compare

quietstats comparedFeature comparisonvs Google Analyticsvs Plausiblevs Fathom Analyticsvs Simple Analyticsvs Umamivs Matomovs Cloudflare Web Analyticsvs GoatCountervs Piwik PROvs PostHogvs Microsoft Clarityvs Pirsch Analyticsvs MixpanelGoogle Analytics alternatives

Learn

Website analytics guidesMetrics handbookAnalytics glossaryPlaybooksGDPR and cookie-free analyticsInstall on SquarespaceInstall on WixInstall on GhostInstall on Google Tag ManagerBlog

Playbooks and tools

Script size calculatorCookie banner checkerPlaybook: Local businessPlaybook: Online shopPlaybook: Blog or publisherPlaybook: SaaS or appPlaybook: Charity or community groupPlaybook: Agency with clientsUTM link checkerConversion rate calculatorCookie banner traffic-loss calculator