Are you running A/B tests the right way?

We’d like to make a wager…we bet you’re not running A/B tests the right way. In case we’ve already lost you (“What the heck is A/B testing??”), A/B testing is a method of testing through which marketing variables are compared to each other to identify the one that brings a better response rate. 74% of companies who take a structured approach to conversion (and that includes A/B testing) have improved their sales.

The basic process of A/B testing involves three steps:

1. Defining your goal and hypothesis
2. Setting your Control (that’s the “A”) and your Treatment (that’s the “B”)
3. Analyzing your results

If you’d like even more of a warm-up to A/B testing, check out our blog post on “How to Run Effective A/B Testing.” A/B testing is an extremely useful tool in your marketing toolbox, but there’s some guidelines you should follow to ensure that you’re using this tool to its fullest capacity and running A/B tests the right way. Bring your A/B testing to the next level by making sure that you’re following these rules. 

Running A/B Tests the Right Way

Decide What Element to Test

Choose only 1 element. Choosing more than one muddies the waters – how do you know what element caused the increase in traffic or clicks or conversions?

There’s a number of different elements that you can test. Here’s a quick list to get you started:

• Subject line of an email
• Offer
• CTA
• Images
• Links
• Content Placement
• Headlines

Note that when you’re testing emails, the open rate tells you the effectiveness of your subject line, while the click rate tells you whether people are engaging with your emails.

Know What You’re Testing & Why

You’ll want to have a clear end-goal. And equally importantly, you’ll want to have a clear hypothesis. What element will improve your click-through rate? And remember – only test (and, therefore, hypothesize on) one element!

Choose a big, bold element to change.  Make a bold hypothesis. For example, hypothesize that the background color of your email will improve click-throughs, rather than the text of the imagery’s caption. See what we mean?

And another thing: focus on testing frequently sent emails or global factors on your website. If you run A/B testing on an email that is sent out annually, how will you know if the results even apply one year from now?

Always Be Testing

How often should you run A/B tests? You should never stop A/B testing! There’s always another element that can be tested, whether it’s on a landing page, a CTA or an email. Once you’ve finished running the A/B test, act on the results. Make the changes if necessary, and, if no changes are necessary, move on to the next A/B test.

Run your A/B tests until you have enough results to be statistically significant. When have you reached a statistically significant number? When you have 1,000 results. Yep, 1,000 is the magic number. Keep the test running until you’ve reached that number, no matter how long it takes.

Are you running A/B tests the right way? If you weren’t before, you will be now! Follow these guidelines and you can be confident – your A/B tests are outta this world.