10/10/80 Tests
To send a test bulletin to a small sample of your audience prior to sending out an important email, we suggest sending the first version to 10% of your subscribers, the second to another 10% of your subscribers that were not sent the initial email, and then sending out the “winning” version to the remaining 80% of your subscribers that were not sent either of the prior two emails. To accomplish this, please follow the directions below.
Step 1: Send out the first version of your email. Use the Random recipient selection filtering criteria type to send the first version to 10% of your subscribers.
Step 2: Send out the second version by applying two criteria. First, use the Random recipient selection filtering criteria type and select 10% of your subscribers. Then add a Bulletin send activity filtering criteria type to select subscribers that were not sent the first version email you just sent out.
Step 3: Now, you'll want to see which version of your email performed better, to decide which copy to send to the rest of your subscribers. You'll need to wait a few hours to be sure the recipients in your test groups have had the opportunity to receive and read your email. Then, go into Reports, and click Bulletin Analytics Report. Then click the Details tab to see a list of your most recently sent bulletins. Here, you can view the open and click rates for both of your test emails. If you don't see open rate and click rate in your columns, click on the Columns link to add more data to this view.
Once you determine which email performed better, per your filtering criteria, click on that bulletin from the Details tab. Under the Sent To: line, you will see the option to Create Copy for Resend.
Step 4: To send the higher-performing email to the remaining 80% of your subscribers, you'll want to use two more Bulletin send activity criteria. You'll send your better-performing email to subscribers who were not sent either of the first two versions of your email.
Testing your bulletins this way can lead to better engagement and overall results for your organization. While it takes a few more steps, the performance improvements are often well-worth it.
50/50 Tests
You can also perform 50/50 tests using filtering criteria. A 50/50 test is one where you send half your recipient list one version of your bulletin, and then send the remaining half a second version of the bulletin:
Step 1: Send out the first version of your email. Use the Random recipient selection filtering criteria type to send the first version to 50% of your subscribers.
Step 2: Send out the second version by applying the Bulletin send activity filtering criteria type to select subscribers that were not sent the first version email you just sent out. This will send the bulletin to all the subscribers -- 50% -- that didn't receive the first bulletin.
Step 3: Again, after a few hours to be sure the recipients in your test groups have had the opportunity to receive and read your email, go into Reports, and click Bulletin Analytics Report. Then click the Details tab to see a list of your most recently sent bulletins. Here, you can view the open and click rates for both of your test emails. If you don't see open rate and click rate in your columns, click on the Columns link to add more data to this view.
