Anthony McGrath

Home › Services › A/B testing

A/B testing

A/B tests that give you real answers

Most A/B tests end too early, test things that do not matter or get read wrong. Tests here are built on evidence, sized for your traffic, run to a reliable result and explained in plain English, so every test teaches you something.

Plan your testsWhat's included
  • 22 years in digital marketing
  • Hands-on: you work with me, not a junior
  • UK, US, Dubai and Australia

By Anthony McGrath · Updated

What does A/B testing and experimentation involve?

A/B testing and experimentation covers test ideas, set-up, running, reading results, rolling out and platforms. Calling a winner after a few days, testing on too little traffic or watching the wrong number leads to changes that look good on paper and do nothing in reality. Anthony McGrath does the work himself, with 22 years in digital marketing, for businesses in the UK, the US, Dubai and Australia.

Best for
Websites and online stores that already get steady traffic
What you get
Test ideas, Set-up, Running, Reading results, Rolling out, Platforms
First step
Measure: analytics, heatmaps, recordings and a walk through your site as a customer, to find where people drop out.
Where
The UK, the US, Dubai and Australia, working remotely across time zones
Who does the work
Anthony McGrath, hands-on, with 22 years in digital marketing

Testing is only useful if you can trust it

Calling a winner after a few days, testing on too little traffic or watching the wrong number leads to changes that look good on paper and do nothing in reality. Worse, real winners get thrown away.

Every test starts with a clear hypothesis from your data, a sample size worked out in advance and one main measure agreed before it starts. That way the result means something.

Before and after

An example of a quarter of well-run tests on an online store.

  • Tests run: 3 (all stopped early) before, 9 (all to a result) after
  • Winning tests rolled out: 0 reliable before, 4 after
  • Conversion rate: 2.0% before, 2.6% after

An example of the kind of change this work aims for, based on typical starting points. It is not a promise or a specific client result: every business starts somewhere different.

What's included

Everything from the test idea to rolling out the winner.

Test ideas

  • Hypotheses from analytics and research
  • Ranked by likely impact
  • Clear success measures
  • A testing roadmap

Set-up

  • Variants designed and built
  • Testing tool set up
  • Tracking checked before launch
  • Quality checks on every device

Running

  • Sample sizes worked out in advance
  • Tests run for full business cycles
  • No peeking and stopping early
  • Problems spotted fast

Reading results

  • Revenue and conversions, not just clicks
  • Results by device and channel
  • Honest calls on unclear results
  • What was learned, written down

Rolling out

  • Winners built into the site properly
  • Testing scripts removed after
  • Follow-up test ideas
  • A library of past tests

Platforms

  • Shopify and WooCommerce
  • WordPress, Webflow and custom sites
  • Landing page builders
  • Server-side testing where needed

How it works

  1. MeasureAnalytics, heatmaps, recordings and a walk through your site as a customer, to find where people drop out.
  2. PrioritiseA ranked list of fixes and test ideas, scored on likely impact and effort.
  3. TestChanges tested properly (A/B where the traffic allows), so you know what worked and why.
  4. Roll out and repeatWinners go live, losers are learned from, and the next round starts.

Platforms and tools

  • VWO
  • Optimizely
  • Convert
  • AB Tasty
  • Shoplift
  • Intelligems
  • GA4
  • Google Tag Manager
  • Microsoft Clarity
  • Looker Studio

Questions

How much traffic do I need for A/B testing?

It depends on your conversion rate and how big a change you want to detect. As a rough guide, a page needs a few hundred conversions per variant to give a reliable answer.

Which A/B testing tool should I use?

It depends on your platform and budget. VWO, Convert and Optimizely suit most sites; Shoplift and Intelligems work well on Shopify.

How long should a test run?

At least one to two full weeks, and until the planned sample size is reached, so weekday and weekend behaviour are both included.

What if I do not have enough traffic?

Then research-led improvements measured before and after are a better use of time than tests that will never reach a result.

Anthony McGrath

About Anthony McGrath

Tip of the day

Avoid tiny split audiences. Combine similar ad sets so Meta gets enough conversion data to exit learning phase.

A UK-based freelance digital marketer with 22 years across Shopify, WordPress, WooCommerce, Magento, conversion rate optimisation, funnels, Google, Meta and TikTok ads, SEO and tracking. He works directly with businesses in the UK, the US, Dubai and Australia, so the person who plans the work is the person who does it. More about Anthony or connect with him on LinkedIn.

Start testing what matters

Tell me your traffic and goals. You get a testing plan sized to your site.

Plan your tests