I’ve had the opportunity to speak and teach on email marketing best practice to audiences of everything from beginners to experts over the past 10 years, and regardless of the experience-level of the group, I often get questions like these:
- What is the best day of the week to send email?
- How many times a month should I email my subscribers?
- What kinds of offers work best to sell online?
And regardless of the experience-level of the group, my answer is almost inevitably, “A/B test it, because every business and subscriber list is unique.”
I’ll give you a few examples:
In my career on the agency side, sending email campaigns on Mondays was avoided like the plague until, in a state of absent-mindedness, a client neglected to push the “Schedule Campaign” button for their weekend launch. The event invitation they had hoped to send on a Saturday (for an event the following Saturday) couldn’t just NOT GET SENT, and in an act of desperation they decided to send it Monday morning. The email campaign had a significant lift in open rates over historical Saturday sends, and the event sold out that day.
For years, I counseled clients to focus on keeping their email content short and sweet. But when I finally convinced a fairly set-in-their-ways high-end winery to A/B test their letter-style (read, epic-length) release campaign against a simple postcard, guess which version was the winner?
(hint: I was wrong)
Best practice is great for guiding decision-making in a pinch, but you can execute a more optimized strategy for your business’s email marketing by adopting one of my favorite acronyms:
Want to get better email marketing results? Curious about what works and what doesn’t with your subscribers? Do you even strategy, bro?
The ABCs of ABT
Start with a hypothesis: What is it that you want to know?
The dictionary defines a hypothesis as “a supposition or proposed explanation made on the basis of limited evidence as a starting point for further investigation.” You want to start with an idea of what you want to prove with your a/b test. For instance, your hypothesis could be that weekend email campaign sends will have a higher open rate than weekday sends because your customers are in more of a buying mood. Or your hypothesis could be that a red button will generate higher click throughs than a green one.
Chose a KPI (key performance indicator) that would identify the winning variant in your test
Don’t run a test if you aren’t sure what “winning” would even look like! In the button example used above, the winner would be determined based on the version that got the most clicks. Therefore, your KPI would be CTOR (Click through of open rate). If your hypothesis is that evening sends will drive higher sales, then your KPI would be your total revenue.
Test one thing at a time
Isolate the variants exclusively to the elements that might impact the KPI you’re trying to improve, so that you have a true A/B test. If you’re trying to A/B test subject line impact on open rate by incorporating a seasonal pull vs. a pricing incentive, don’t also change up the email content between the two versions of the hero image, or send the two subject line variants at different times of day!
Apply your findings to your next campaign and test something new!
Once you’ve identified a winning element of your email formula, stick with it for a while and focus on testing other elements to increase your success. Document what you’ve learned, both for your own reference and so that it is easy for somebody stepping into email marketing roles and responsibilities down the road to identify how you arrived at your winning formula.
What should we test?
You might not be A/B testing your email marketing campaigns yet because you simply don’t know what you should be testing. Or perhaps you have a wealth of ideas for what to test, but don’t know what order to do it in!
This wouldn’t be an ultimate guide if I didn’t walk you through the most high impact A/B tests you can run for your business. And I’m upping the ante by giving you 12 A/B tests to run (plus some bonus tests, because I’m crazy like that) in this downloadable cheat-sheet. Booya.
What order should I test in?
I like to think of this in terms of your subscriber funnel. Envision your entire database at the top of your funnel, and the primary goal of your campaign (whether that be RSVP, an email response, or a conversion on your website) as the bottom. Along your subscriber’s journey towards the bottom of the funnel (your campaign goal), there are specific actions they’ll need to take in order to arrive at the desired destination. When considering the order you’ll want to stack your A/B testing, think of the actions you can impact by the test and work from the top to the bottom.
Optimize for open rate:
Open rate is the percentage of your delivered emails that get opened by the subscriber. Shoot for an open rate in the 30s. Anything below that, and you might not have a very engaged database. Anything above that, and I will hug and high five you when we meet in person.
To optimize your open rate, A/B test the elements that drive your customers to open your email: things like timing, the first impression in their inbox (subject line and preheader text), and even the sender ID of your campaign.
Day of week
weekend vs. week day
THEN
week day 1 vs. week day 2 or weekend 1 vs. weekend 2
There can be significant differences in your customer’s engagement level based on the day of the week, as evidenced by the story I shared earlier about my client’s event invitation.
If you’re reaching out to an audience that is predominantly business people, odds are they’ve given you their work email address, and fingers-crossed they aren’t checking that email on the weekend.
On the flipside, you may be selling a commodity, lifestyle product or luxury good, and your audience might be feeling more free and loose about their disposable income on Friday, Saturday or Sunday.
These are simply general concepts for you to consider in structuring your weekend vs. weekday A/B test. You can also refer to historical email sending data to find a few days that seem ripe with opportunity to run a head to head test. Based on the results you can move to the next Day of the Week test and give your weekday or weekend winner further vetting by testing Saturday vs. Sunday (if weekend was the winner) or a couple of weekdays against one another.
Time of Day
am vs. pm
THEN
pm vs. pm or am vs am
THEN
mid morning vs. mid afternoon
There are 24 hours in the day, so the combinations here are many. However, start with the most extreme differentials – such as 6 am vs. 6 pm, then, depending on the results you can go further into working hours vs. non-working, or mid-morning vs. mid-afternoon if you are seeing trends that push the results into the middle of the day.
Subject Line
personalization yes/no
emojis
offer vs. sentiment
question vs. statement
If the 24 hours of the day create more combinations of A/B tests than you can keep straight in your head, the number of words in the English language that you can test in your subject lines is enough to make your head spin. Here again I encourage you to consider the high-impact elements to test.
Try calling your subscriber by their first name to capture their attention:
Erica, your invitation is inside
Vs.
Your invitation is inside
Emojis are another way to grab a subscriber’s attention in a crowded inbox. You will want to assess whether the use of emojis will fit your brand voice and tone, but if you are feeling playful you could definitely test how effective emojis are in getting your emails opened.
Testing an offer vs. something more sentimental is another way to see what drives your subscribers to take action. Are they motivated by a pricing incentive, or more interested in knowing what their perfect summer outfit might be? A/B testing can help you discover the formula here.
Save 25% on perfect summer outfits
Vs.
Your perfect summer outfit is inside
Asking questions or making bold statements are another great way to instill curiosity in your subscribers and drive them to open your email.
Looking for the perfect gift?
Vs.
The search for the perfect gift is over.
Optimize for clicks/responses:
A subscriber must open your email campaign in order to take the action you want them to take, but once you’ve tested key elements to improve your open rate, you can start running A/B tests to optimize your content and get your customers to act.
The real estate in your email campaign that will get the most subscriber attention is the content placed “above the fold”.
“Above the fold” is a holdover from the days of newspapers (and dinosaurs), when publishers wanted to make their biggest splash with the content that was presented above the fold of the newspaper. In terms of your email campaign design, this is usually the top 3rd, or the content that displays before a subscriber has to scroll.
HOT TIP:
“Above the fold” is going to vary based on the size of the screen your email is being viewed on. What lands above the fold on a large desktop might not necessarily make the cut in on an iPhone or Android screen.
To get the most impact from your A/B tests on email content, test elements ABOVE THE FOLD. I recommend testing headlines, calls to action, and hero images.
For headlines, you can capitalize on winning elements from your subject line tests on sentiment vs. offer, or asking questions vs. making bold statements. You can also test the size of your headline and offer text, or using images vs. simply letting the live text do the talking.
Speaking of images – what about testing the kinds of images you use in your email creative? Try testing an image with people in it vs. a more product-centric image or seasonal and setting-driven images. Depending on your business and the kinds of products you sell, this can get pretty complex or stay very simple, but it isn’t to be overlooked. If a picture paints a thousand words, and people don’t read much, having the right kinds of images in your email campaigns can be the difference between sales and goose-eggs.
Another truly high-impact element to test is your CTA (Call to Action). The words and the design of your CTA can help catch the subscriber’s eye and get them to click. Start with the overall design – is your CTA a button, or do you use bold, colored text? Test the color you use as well – does a red button get more clicks than a gold one?
What words do you use to drive customer action? It should be clear, and set the customer’s expectation for the next thing that will happen. DO use words that help the customer understand that they will be leaving their inbox or opening up a draft email by clicking…
Test the verbs that you use, e.g. Shop vs. Buy vs. Save, or Acquire vs. Secure. Test the first person, e.g. “Download my cheatsheet” vs. “Download the cheatsheet”
And what else?
While I like to focus my energy for A/B testing on the above the fold elements, you could also test entire email formats to get really obvious results on what moves the needle for your audience. For instance, you could test a super simple postcard-style email with minimal text and a strong call to action against the same offer presented in more of a letter format. You could test a banner image + text against a full-length hero image + text.
Summin’ it up
A/B testing in your email marketing campaigns is a simple, low-cost way to get valuable marketing insights on what does and doesn’t drive your customers to action. Leverage the A/B testing functionality within your email service provider to test high-impact elements that effectively drive your subscribers toward your campaign goals, and you’ll get smarter with every email campaign you send. I’ve taken the guesswork out of A/B testing your email marketing campaigns by telling you what to test and in what order – all you have to do is execute!
By downloading the Email Marketing A/B Test Cheatsheet, you’re getting the equivalent of a full year of A/B tests to help you get laser focused with your email marketing strategy. Never waste an opportunity to learn from your email subscribers, get nerdy with the data, and celebrate your successes with your team (and here with me, in the comments, if you’re willing)!
Are you ready to ABT?
ABT, FTW!