11 Tested Tips to Improve Open Rates Through Email A/B Testing in 2023

As marketers, it is not uncommon for us to become overly engrossed in our own perception of what we believe to be visually appealing, often overlooking the importance of considering our audience’s preferences.

Consequently, we find ourselves in situations where we confidently initiate an email campaign that appears to be promising, only to encounter disappointing results.

Countless hours were dedicated to meticulously crafting an impeccable template for email marketing, ensuring an ideal call-to-action (CTA) button, and selecting the perfect image.

Nevertheless, much to our dismay, we discover that our target audience remains disinterested in the content of our email. At this point, we are left wondering: where did we go wrong?

You can use A/B testing to pinpoint what’s working (and what’s not).

In this blog post, we’ll show you:

What email A/B testing is
How email A/B testing works
Why email split testing matters and the benefits
Eleven fantastic tips to boost your email open rates through A/B testing.

Let’s get started!

Table of Contents

What Is A/B Testing?

A/B testing compares two versions of a webpage or applications against each other by tweaking a single element to determine which one has the better performance.

In essence, you have version A and version B — and you tweak one element of your email marketing campaign at a time.

Keep in mind that just because something is newer doesn’t always mean it’s improved. You have to be willing to listen to the results and let them drive your decision.

Okay, so what should you split test? Well, that’s mainly up to you. Perhaps you want to pit two versions of your homepage against one another. You can also test your landing pages or the look of an app.

Sometimes, it’s not about testing the entire page but certain elements.

For instance, does font A or font B look better? Should you go with a bright color scheme or one that’s darker and more muted? Does the CTA button drive more results at the top of the screen or in the center? Should it be blue or pink? What about the wording?

Here’s an amazing video that explains the science behind A/B testing:

It’s always better to tweak a single element at a time to gauge the performance difference — otherwise, how can you pinpoint which change performed better?

How Email A/B Testing Works

If you’re not already using A/B or split testing, you should start now.

One roadblock that may hold you back is your lack of understanding of how split testing works.

It sounds so easy, but what kinds of tools and software do you use to test the control against the variation? Who decides which one is the winning variant?

Those are all perfectly valid questions.

In this section, we’ll take a basic example, two versions of an email, and use split testing to determine which would have the best results.

Here’s how to begin a test email campaign with four popular email platforms: EngageBay, Campaign Monitor, AWeber, and MailChimp.

#1. EngageBay

The all-in-one sales and marketing platform EngageBay is an excellent resource for split-testing.

EngageBay email ab testing — *EngageBay email A/B testing feature*

EngageBay’s A/B testing works as follows. You write several versions of your email subject line, then use the Email A/B Testing Feature.

You can test the headlines within that feature to determine which one would generate the highest open rate.

👉 Boost your email marketing campaigns with EngageBay’s customizable email templates tailored for engagement and success.

#2. Campaign Monitor

If you use Campaign Monitor for your email marketing, then you’re one of about 250,000 people and counting.

Campaign Monitor has its A/B testing in email marketing built right in. You can compare content such as the email body, the from name, and the subject line.

Pick one to start with.

Campaign Monitor email ab testing — *Source: Zapier*

You send between 20% and 30% of your overall email marketing list both versions of the email.

You even get to select the conditions that determine the winner: people who click on the link, the total unique clicks, or the open rate. You get to pick the length of the test as well.

#3. AWeber

Aweber is another excellent email marketing software with A/B testing tools.

You will need a contact list of at least 100 people for AWeber’s most accurate split testing results, so make sure your list is up to par.

To begin testing, choose the Broadcast Split Tests option. Make sure you only use two variants instead of four.

Select how many contacts you want to receive the message as a percentage. Then you can use AWeber’s HTML coding, plain text messaging, or drag-and-drop email builder.

From there, pick the subject line or another part of your email you want to test. It’s your job to make both emails identical save for the element you’re split testing. Once your emails go live, AWeber sends you the results.

This tells you who wins. Using AWeber for A/B testing is somewhat more complicated than Campaign Monitor, so keep that in mind.

#4. MailChimp

You can also use MailChimp for your split testing needs.

They suggest using 20 to 50 percent of your audience for split testing purposes. Then, you must choose what the winning metric is, such as higher open rates, click rates, or views. You again get to determine how long the testing will run.

To wrap up, include the testing information in the form fields. Then your emails go out. You can do this yourself or let MailChimp automate the email sending. Once you know how your recipients respond to your campaign, you can use the test result for your future email.

The Benefits of A/B Testing Your Email Campaigns

How hard does your small business currently work to obtain more customers? The money you’d spend on A/B testing is a fraction of your current budget, yet split-testing can be an assured method for increasing open and click-through rates.

Another perk of A/B testing: It tells you what works so you can do it again. Sometimes, getting your email campaigns right feels like there’s a bit of luck and magic involved.

Not so with split-testing. You’ll know whether your newly-positioned CTA worked or if it was that awesome subject line. You can then replicate these tactics in future email campaigns.

Enhance Your Email Marketing

Want to make your emails more impactful? Check out our beautiful, easy-to-customize recruitment and marketing email templates. Designed to boost engagement, these templates from EngageBay will help your emails stand out. Just customize the images, headings, and CTAs for your brand, and hit send in a few minutes!

Browse the Template Library

Why Does Email A/B Testing Matter?

Indeed, it is understandable that A/B testing can appear time-consuming, leading to the question of whether it can be bypassed altogether.

However, it is essential to recognize the significance of A/B testing in the context of email marketing and the considerable impact it can have on the success of your business.

The core reason why A/B testing matters lies in the fundamental importance of email marketing itself.

Emails serve as a powerful channel for communication, allowing businesses to directly engage with their target audience.

However, if the emails sent to your audience consistently go unopened or are swiftly deleted, it not only signifies a waste of your company’s valuable time, effort, and manpower but, most significantly, a squandering of financial resources.

For small businesses, every penny counts, and investing resources into initiatives that do not yield profitable returns can be detrimental. A/B testing plays a crucial role in mitigating such risks by providing insights into what truly resonates with your audience.

By systematically testing different variables within your email campaigns, such as subject lines, content, imagery, or even the timing of delivery, you can gather invaluable data on what works best.

This data-driven approach enables you to make informed decisions, optimizing your email marketing strategy and maximizing its effectiveness.

By skipping A/B testing, you would be left with a significant degree of uncertainty regarding your email campaigns.

You would be relying on assumptions rather than concrete evidence, which can lead to missed opportunities and poor results. A/B testing allows you to gain valuable insights into your audience’s preferences, allowing you to fine-tune your emails and subject lines to better resonate with their needs, desires, and pain points.

About 77 percent of businesses run A/B tests on their websites (including landing pages) to identify design, font, and other such issues. This helps reduce cart abandonment by highlighting what causes buyers to abort a cart. There can be a variety of reasons such as a poor layout, hidden costs, etc. (Adoric)

Email A/B Testing Rules to Follow

For the success of your split testing, make sure you follow these rules and best practices:

Depending on the type of company you run, you’ll focus on different testing metrics. CTAs, homepage messaging, free trial signup flow, and lead form components matter a lot for tech businesses.
If you’re in e-commerce, pay special attention to testing your funnel components, navigation elements, and homepage promotions. This helps you boost your sales over the holidays and the average order volume and checkouts.
For travel businesses, A/B test for such metrics as ancillary product presentations, search results pages, and homepage searches.

Media companies strive to boost their social sharing while growing their readership. They also want to maintain the readers they have, keeping them on the site longer. It’s important to run split tests on social sharing buttons, recommended content, and email sign-ups to meet your goals.
No matter what industry you’re in, when you’re done testing, make sure you remove remnants of the test. The longer these remain on your site, the better the chances of the Google algorithm picking upon them. This can appear deceptive to both Google and other search engines. When your test ends, leave no traces.
On a similar note, make sure you’re not testing for a very long time, either. For instance, there’s very rarely any need to test for more than 24 hours. You’re not reaching out to your entire contact list. Those who don’t respond won’t, but you’ll still get good enough results if most people participate.
Source: Ceralytics

To make it easier to get rid of your split testing pages, make them 302 redirects. These are temporary by design. If Google or another search engine picks up on these through their algorithm, they know these pages won’t stick around forever. They also won’t index this temporary page. Compare that to a 301 redirect, which is permanent and can thus get indexed. If both split test pages are set as 301 redirects, this can hurt you.
The rel = “canonical” attribute can also help to this end, especially if your test has more than one website URL. This attribute tells Googlebot that all the URL variations should go back to your main page, which was there originally. This also prevents the algorithm from dinging you.
Source: Quora
You shouldn’t cloak either, which is when you limit the content to a search engine. The customer or test participant can see the full range of content, but a search engine cannot. Limit the segmentation you do by IP address or user agent address, so you’re not at risk of cloaking. If you’re caught doing it, you could face the consequences. Your page could disappear on any results pages through Google or other search engines.

11 Tips to Improve Your Email Campaign’s Open Rates Through Email A/B Testing

Now for the main course: Here are 11 tips to improve your email campaign’s open rates through email A/B testing.

You don’t have to apply everything we recommend — mix and match to suit your needs.

1. Link Count Testing

How many links are in your emails? Is it just one or two, or do you have way more than that?

While we recommend you test this, the fewer links you have, the better. There are two reasons for this.

The first is a psychological one that employs what’s known as the paradox of choice.

Perhaps you’ve heard of the famous jam study. If not, here’s a recap: two groups of people were at a supermarket. Both got a selection of jams to pick from, one with six flavors and the other with 24 flavors.

While people liked the table with more varieties of jam, they didn’t buy nearly as much jam as those with fewer options. That’s because the more choices we get, the harder it becomes to pick something out. In the end, most people will just walk out empty-handed.

The same thing applies to your links. The more you have, the more challenging it becomes for the email recipient to decide which ones to click on. In the end, they may not click on any of them.

The second reason we recommend you cut down on links is for aesthetic reasons. It’s because too many links look spammy. Your email could get stuck in the spam filter instead of the recipient’s inbox.

2. Email Send Day/Time Testing

We mentioned this numerous times in our blogs: Tuesday, Wednesday, and Thursday are your best days to send emails. The ideal times to send emails are early in the morning, midafternoon, and later in the evening.

best time to send emails - email ab testing — *Source: CoSchedule*

Okay, so if that’s all so concrete, why bother with email marketing testing? Well, because those times and days can fluctuate slightly depending on your audience.

Even if you think you know the best days and times your audience is most receptive to your emails, it could change. This will always be in flux, and you have to be prepared accordingly.

3. Email CTA Testing

Ah, the call to action. It’s so necessary, but is it working for your company?

Your email CTA can appear in several formats. Perhaps it’s a line of text at the end of your emails. Maybe it’s the centerpiece of your email body. There’s proven research that shows the CTA button is more effective than a link in emails. According to data from Campaign Monitor, it’s possible to see a 28% boost in your click-through rate with a call to action button.

That might be the first thing you test: buttons or links?

Once you make that decision, you’re not done yet. You should also test for the following:

The amount of whitespace your email has, which can either obscure your CTA button or let it stand out
The color of your CTA button, as well as the other colors in your email, including the background
The design of the button, including shadows, edges, and more
The size of the button
The placement of the button
The CTA button copy

4. Anchor Text Testing

Are you using anchor text in your emails? If not, you’re missing out on a great way to pique the interest of your audience.

What exactly is anchor text, anyway?

This is a hyperlinked copy that often ends in an ellipsis. It’s a preview of what’s to come and draws out their natural sense of curiosity. Here’s an image that showcases how anchor text can play to several emotions.

Emotions in email ab testing — *Source: Yesware*

The first option you have is to get your audience curious. If this is the approach you take, then the accomplishment you highlight should be something they want to do. Make sure you don’t give the whole secret away in your anchor text.

You can also invoke empathy in your audience, but you’d have to know their pain points to do this.

Maybe your audience is having a tough time converting leads or getting anyone to read their blog. By showing you’ve had the same problems and overcame them, you display your empathy.

Finally, you can invoke a sense of fear by making them feel like they’re missing out on something extraordinary. That’s what FOMO is all about, after all.

No matter which of the emotions you go for in your anchor text (and maybe it’s another emotion entirely), remember this is just a preview. Once the reader clicks the link, they’ll get the whole story.

Discover the most effective opt-in email strategies that will transform your subscriber list in our detailed article!

5. Personalization Testing

You know you should personalize your emails, but how much personalization is enough? Only A/B testing can answer that question. Here’s a great example of a travel company. You can see two personalized emails they tested side by side.

In the first version of the email, it isn’t super personalized. Yes, it says “featured hotels just for you,” but that’s about as far as it goes.

On the other side of the email are all the exotic locations a customer might want to travel to. It’s a good email, but is it great?

Not as great as the second email. This one has the name of the email recipient outlined in the sand. Their name is also in the body’s first line beneath the image.

How excited would you be to see your name traced in the sand? Very, right? That alone is going to make you want to read through the email.

Get creative with your personalization — and then A/B test it.

6. Tone Testing

How do you write your emails?

Perhaps you take on a more informal tone, such as the voice of this blog post. It’s relaxed yet informative and feels like talking to someone you know. You’re reaching out to your email audience on a more personal level.

Maybe you prefer the formal approach. The writing is more buttoned-up and may include jargon and terms that only those in your niche would understand.

Both are appropriate in some instances and industries.

Also, your tone might vary depending on which audience segment you’re trying to reach. You might write in a more relaxed tone with your established customers and more formally with your new ones.

The opposite could also be true.

Split testing is the best way to know what your audience likes best.

Prepare two versions of the same email, one with an informal tone and the other written formally. Then see which one gets the highest open rate.

Testing for email tone is not a one-and-done thing. You’ll have to keep repeating the test when sending major emails to new and established customers.

7. Content-Length Testing

No one likes slogging through a never-ending copy of an email. Going in the opposite direction can be risky, too, since brevity can also be vague if you’re not careful.

We recommend testing at least three versions of an email. One should be the long one you prepared with love and time.

The other should be a mid-length version that’s shorter but not too short. Finally, condense your message into the lowest number of words possible.

Now test all three emails to see which gets the highest open rates. We’d bet it’s probably the mid-length one. That’s not to say that longer emails never have their place. For example, in a newsletter email, your audience expects a longer, meatier message.

Short emails are also appropriate in some instances. If you’re sending a reminder about an upcoming event or webinar, short messages are more appropriate.

Are you having a hard time cutting your emails down to size? You could always try adding some images to break up longer blocks of text.

A study by HubSpot found that many consumers (65 percent) liked seeing images in emails more than they did text.

Have you written about the subject before in a blog post? Then link to that rather than devoting paragraphs to the same topic in your email.

This lets the audience choose if they want to read further. Just remember not to add too many links.

8. Link Text Length Testing

Speaking of links, there are so many ways you can format them.

For instance, let’s say we’re linking to our EngageBay homepage. We could just say “check out EngageBay for all your marketing need” and link to the company’s name. Alternately, we could do something like this: “if you need an all-in-one sales and marketing software.”

When you add hyperlinks to your email, how should you do it? How many words should be included in the links? This is something else you’ll have to test.

Most marketers agree that 12 words are the maximum amount. Here’s a MailChimp illustration that shows that data in more depth.

email ab testing statistics — *Source: Yesware*

It appears the more words you cram into your email links, the lower and lower the email open rate is.

9. Greeting Testing

Did you know the very opening words of your email can be a big determinant of your email campaign’s open rate success? It’s true.

Whether you say “hello,” “howdy,” or “good morning” does matter. Use A/B testing to determine what types of openings influence your readers the most.

Like the body of the email, the tone of your email greeting will vary.

If this is a new customer and you are keeping a somewhat formal tone, you might write “welcome” as your greeter. Starting your message with a “hey there” is perfectly appropriate for older, established customers.

What seems to matter most in terms of open rates is greeting personalization. Instead of “Hello,” try “Hello, [name].”

10. Subject Line Copy Testing

One of the most important parts of any email campaign is the subject line. After all, this is the first impression you make on your email recipient.

Your subject lines have to be good.

There are lots of ways to craft a fantastic subject line.

Personalization in subject lines is always good. Stats and figures can pique interest. Discounts and deals get most people clicking.

Your company can even use emojis in the subject lines — but don’t overdo it.

Emojis email ab testing — *Source: Marketingprofs*

Whatever works for you, you have to know for sure. An A/B test will truly come in handy here.

Like your anchor text, your subject line can connect with your audience on an emotional level. You can use empathetic subject lines to connect with your readers.

You can also foster that sense of FOMO by sending an email about a deal that’s only good for today or a sale that’ll last 12 hours more.

You can get your audience curious by writing a subject line much like you would your anchor text — by making a partial statement, ending it with an ellipsis. By telling only part of the story in your subject lines, you hopefully intrigue your reader enough that they want to click, open the email, and see how the ending goes.

11. Subject Line Length Testing

Okay, so you’re confident the content of your subject line will appeal to your audience. You A/B tested it and everything. You’re all ready to send the email, right?

Not so fast.

You should also test for the length of your subject lines before you launch your email campaigns.

Exceeding 50 characters is no good for your subject line, as there’s a chance the audience member might see a cut-off version of your subject line. This depends on the device they’re using, of course.

Your audience is more likely to see the crudely chopped subject line on smartphones.

Five words are the sweet spot for subject lines. This should fit the character count requirement so that even if your audience is on their phone and holding it in portrait mode, they should see the whole subject line.

Conclusion

The implementation of A/B testing in your email marketing endeavors holds tremendous significance in ensuring the success of your email campaigns.

By initiating an A/B testing campaign, you can ascertain that every single element incorporated resonates effectively with your target audience.

No aspect is too insignificant to undergo split-testing; from the strategic placement of your call-to-action (CTA) button to determining the optimal time to dispatch your email, each detail is worth meticulous experimentation.

While some components of your email campaigns may necessitate regular testing, others may only require a few rounds of evaluation. By gaining a deep understanding of your audience, you are empowered to craft stellar and triumphant campaigns.

A/B testing acts as the indispensable tool that guarantees all facets of your campaign are finely tailored to your audience’s interests and pain points, thereby securing a devoted and lucrative customer base.

The commitment to A/B testing not only enhances your campaign’s performance but also serves as a powerful catalyst in cultivating a legion of loyal customers who actively engage and make purchasing decisions.

The great thing is you don’t need to do everything manually. For small businesses, EngageBay is an excellent option. It lets you A/B test your email campaigns by letting you create a variant and then test it against your main campaign. You can choose the winner, and voila: your open rates go through the roof!

11 Tested Tips to Improve Open Rates Through Email A/B Testing [2023 Updated]