Research: The best time to send your newsletter. Download now >

Successful with
a/b testing

Easy way to increase the performance of your newsletters

In order to easily maximize opening and click rates of your email marketing newsletters, simply chose the right subject line or sender address. However, is it as easy as desired? How can you find the right subject line or sender? Based on the numerous requirements made to a subject line (the shorter the better, inviting, curious, personally appealing, etc.) you can use the practical mailworx feature to test your variant diversity with an A/B test. You simply identify two possible subjects and mailworx will decide for you which one is more successful.

By using the A/B split test you can gain up to 2.5 times higher click rate and a 50% higher opening rate. That's what measurements have shown over millions of emails - try it for your own and convince yourself.

 

Questions

What is an A/B split test?

How does an A/B split test work?

How can I use the A/B test in mailworx?

Which criterion does the split test need?

How should the test group be divided?

When will the winner be determined?

When should the split test start?

How can I see how successful the A/B split test was?

 

If your question hasn´t been answered in this whitepaper, please let us know. We´d gladly catch up on this.

 

Ask your question

 

What is an A/B split test?

The A/B split test in mailworx enables you optimizations of your email marketing performance by testing which subject line or sender address achieves the best opening or click rate. The subject and sender of a newsletter are the most important criteria for deciding whether to open a newsletter or delete it. Therefore, a simple mailworx feature can optimize these two parameters.

In general, the A/B testing refers to the comparison of two versions with similar content in order to find out which of these versions is better performing on the corresponding audience. You are working with two test groups and they are provided with one of your newsletter versions. The variant that performs better is sent to the majority of your recipients in order to cause you better results.

 

How does an A/B split test work?

For example, if you want to send a newsletter to 100,000 recipients, the optimal subject line is very important. The A/B split test selects random test groups - for example, to the extent of 10% of your whole subscriber database. The first subject variant is sent to 5% (5,000) of the recipients and the second variant is sent to the other half of your test group. After a defined period of time the system determines which subject line has achieved the highest opening rate (or click rate) and the remaining 90,000 newsletter subscribers are provided with the more successful subject.

The 10% of newsletter recipients are selected randomly and these subscribers also do not know that they are being used for an A/B test. That's why they behave the way they normally would - so it's a representative sample and the results can be applied to the remaining recipients.

 

How can I use the A/B test in mailworx?

It's easy to create an A/B test for your subject line or sender address in mailworx. You can find the appropriate settings in the "send" tab of your email marketing campaign. Simply switch the send type to "A/B Split Test".

 

ab testing

 

In the moment you switch to this send type, mailworx refreshes the interface and the needed settings are available to you.

 

Step 1: Define the attribute for your split test

You can choose between optimizing your subject line or the sender address. With just one email, you have the opportunity to get to know your target group even better. Simply try a subject line with icons or personalization elements - there are no limits to your creativity. Instead of the anonymous address of the company name, the emails may also be sent once with the corresponding customer advisor of the recipient. Try out for your own and let yourself convince of the performance improvements.

 

Step 2: Choose the size of your test group

The test quantity depends on the size of your target group. In most cases, we recommend between 10 and 30 percent of the whole recipients. With a simple slider, you can resize your test group and see immediately how many emails are contained in each subset.

 

Step 3: Determine the time period of your test

When you use to send your newsletter with an A/B test, the large amount of newsletters is always sent after a certain time - consider this, when planning a split test. You have to keep in mind that your test recipients should have the opportunity to open the newsletter and react on your content. We recommend giving your recipients at least one day and mailworx will automatically send the better version of your defined criteria to the rest of your recipients.

 

Step 4: Plan your split test

Last but not least, you have to determine the sending time of your newsletter. You can send it immediately or choose a date to send it later. Keep in mind that first there only will be sent a little part of your emails - the majority of your newsletters will be sent after the winner of the test is identified.

 

How can I see how successful the A/B split test was?

The success of the A/B split test essentially depends on three factors:

  • Is the size of the test group suitable?
  • Is the test attribute (subject/sender) variable enough?
  • Is the waiting period between the test and the real sending sufficient?

Whether the A/B split test was successful and has led to a statistically significant result is indicated by the detailed view of your email campaign statistics.