How to A/B Test the Sending Time of Campaigns in Klaviyo

Joseph Hsieh

February 5, 2021

Sending out your email campaign at the right time can improve your engagement performance. Although there are a number of studies that recommend the best time of the day to send email campaigns, you should remember that there’s no universal time that guarantees the best results for all brands and audiences.

When is the Best Time To Send For Your Audience?

The best time can vary based on the different types of subscribers and their behaviors. Whether your subscribers are professionals who only do their online shopping on their breaks or evenings,  or a younger audience who always have their smartphones on hand, or where they are in the buying lifecycle, etc. All these behavioral differences among the different types of subscribers can contribute to the fact that there’s no universal time of the day for email marketing that will work for all brands. The best way to know is to test and learn.

Can’t I Just Look at My Past Campaign’s Send Time Performance?

Although you can manually test the different sending times by simply sending each campaign at a different time, you can’t get definitive results because of the difference in the content between those campaigns and the numerous external factors that might have happened in between. To ensure that you get the best results, you should test with the same campaign, same subject line, and same day of the week. Basically, keep everything in the campaigns constant except for the sending time, thus removing all other factors from the equation to get more confident results. Luckily, in Klaviyo, there is an easy way to do just this.

Klaviyo’s Built In A/B Testing for Send Times

Klaviyo offers a feature that will enable you to test the sending time for your campaigns. You can set your campaign to be sent at different times throughout the day while randomly selecting subscribers from your targeted audience. You can have two or more variations of the same email within a single campaign ensuring that everybody gets the same content, only to differ in the send time.

Let’s explain how it’s done.

First, you need to prepare your campaign as usual. If you need any help with setting up email campaigns you can check our tutorial on “How to Send or Schedule a Campaign in Klaviyo” to get you started. Once you’ve selected the audience for your campaign and you’ve created the email content, you need to add another variation to start configuring your A/B test for send time.

In the “Message Content” part of your campaign, click on the “Add a variation” link at the bottom of the page as shown below.

how-to-a-b-test-the-sending-time-of-campaigns_1
In the “Message Content” part of your campaign, click on the “Add a variation” link at the bottom of the page as shown below.

This will get you to the A/B test configuration window for testing different content. To switch the configuration window to A/B test for sending time, click on the “switch to test send time” link at the bottom of the page.

how-to-a-b-test-the-sending-time-of-campaigns_2
To switch the configuration window to A/B test for sending time, click on the “switch to test send time” link at the bottom of the page.

Now you should see two variations of the same email, but with the option to set a different sending time for each variation.

how-to-a-b-test-the-sending-time-of-campaigns_3
Now you should see two variations of the same email, but with the option to set a different sending time for each variation.

Klaviyo will automatically set the distribution weight between these variations to 50% each. If you add a third variation, the distribution weight will be changed to 33% for each variation. To add a third variation, click on the “Actions” dropdown menu button and select “Duplicate Variation”.  Note: you can change the different weights if you’d like, but we usually recommend having an even split to get enough data for each variation.

how-to-a-b-test-the-sending-time-of-campaigns_4
To add a third variation, click on the “Actions” dropdown menu button and select “Duplicate Variation”.

Although you can create many variations (we’ve tested up to 15), remember that your targeted audience for that campaign will be split between all variations. This means that if you have 1000 subscribers for this campaign and you create 10 variations at 10% weight, each variation will be sent to only 100 subscribers. Having too few subscribers in the test groups might not get you statistically significant results.

Set a different time of the day for each variation.

how-to-a-b-test-the-sending-time-of-campaigns_5
Set the date and time for each variation

The checkbox below the variations is an option to add a different UTM parameter for each variation in the URL. This is a helpful option if you’re using analytic tools like Google Analytics that uses UTM parameters to let you drill down and see the performance of each variation separately.

When you are ready to proceed to scheduling your campaign, click on the “Save & Schedule Campaign” button in the bottom right corner.

how-to-a-b-test-the-sending-time-of-campaigns_6
When you are ready to proceed to scheduling your campaign, click on the “Save & Schedule Campaign” button in the bottom right corner.

Analyzing the results

After you send your A/B test campaign you need to analyze the results. You need to check the performance of each variation and determine which variation has the best results. Note that you need to give the campaign enough time to collect all the data. Typically, not all the opens will happen on the initial day of sending, so, that’s why we prefer to wait at least 24-48 hours before examining your campaign results.

From the “Campaigns” section, click on the campaign you want to analyze.

how-to-a-b-test-the-sending-time-of-campaigns_7
From the “Campaigns” section, click on the campaign you want to analyze.

On the next page, you should see an overview of the campaign. Click on “Variations” and you should see the variations you set for this campaign with the results for each of the variations in each row.

how-to-a-b-test-the-sending-time-of-campaigns_8
Click on “Variations” and you should see the variations you set for this campaign with the results for each of the variations in each row.

From here, you can examine the Open Rate, Click Rate, and the Place Order information for each variation.

In our Sample Campaign, we can see that Variation C which was sent at 8:00 p.m., has the highest Open Rate and Click Rate. This tells us that the audience for this campaign is most active in the evening and the best results can be achieved by sending campaigns at that time.

Additionally, you can test even further (e.g. 7:00 p.m., 8:00 p.m., 9:00 p.m.) to get even more precise results.

If you have audiences of greater than 12,000 subscribers and you want Klaviyo to help you decide the optimal send time, you can also consider the “Smart Send Time” optimization feature in Klaviyo here.

Recent Posts

How to Disable or Enable the Double Opt-in Process in Klaviyo

What is "Double Opt-in" and how to disable it in Klaviyo.

How to A/B Test the Sending Time of Campaigns in Klaviyo

Learn how to A/B test the sending time of campaigns in Klaviyo.

How to A/B Test Subject Line and Content in Campaign Emails in Klaviyo

Learn how to A/B test your campaigns in Klaviyo.