5 Dos and 5 Don’ts of A/B Testing

A/B Testing – another tool in your marketing toolbox.

Last week, we asked an important question – Are you running A/B tests the right way? That blog was all well and good, but sometimes blogs like that are just too much “meat” to handle at one time. Sometimes, all you’re looking for is a quick summary.

Today, we’re taking that list of guidelines for running A/B tests one step further (or backwards, depending on which way you look at it). Here’s 5 dos and 5 don’ts of A/B testing. By the end of this article, you’ll know exactly what you should and should not do when A/B testing at your company to keep your results in tip-top shape. And, better yet – you’ll know it in less than 300 words from now. 

5 Dos and 5 Don’ts of A/B Testing

Do

1. Test 1 variable at a time.

2. Decide on a statistically significant number for the test results. For email-based A/B testing, allow at least 1,000 people to react before ending the test.

3. Focus on making changes that can have the biggest impact.

4. Pay attention to external factors that can effect your results.

5. Test a variety of elements – imagery, color, copy, offers, buttons, etc. – throughout the months. You can always be testing something!

Don’t

1. Don’t split up your sample groups unevenly – allow the same amount of people to respond to both “A” and “B,” especially when pertaining to email.

2. Don’t vary the timing of your A/B testing – again, especially with email. But, even with landing pages and CTAs and home page changes…make the changes simultaneously. Otherwise, you’re unwittingly adding another element to your test (time).

3. Don’t test multiple assets at the same time. We mean business on this one, which is why you’re seeing it so often here.

4. Don’t be afraid of tests that reveal surprising results – that’s the whole point of running A/B tests. Your hypothesis might be off and the original color choice or image or copy is the best option.

5. Don’t stop testing. Finished one test? Run another!