Black Friday and Cyber Monday are still a few months off, but you should already be prepping your strategy for ecommerce’s biggest weekend of the year.

According to Klaviyo, email performance peaks during BFCM with conversions climbing from a median of $0.05 per received email to $0.07 in November 2022.

However, to maximize your conversion rates during this time, you must first focus on boosting your open rates with solid subject lines.

The efficacy of your emails’ subject lines is crucial all year long, but especially important during Cyber Week. In order to stand out in a flood of marketing emails, you need to know exactly how to grab your audience’s attention and entice them to open.

How? Start A/B testing your subject lines as soon as possible.

I’ve spent years A/B testing subject lines to optimize open rates for over a dozen clients. Here’s what I’ve learned and how you can use it to strengthen your strategy for BFCM 2023.

A/B Testing 101

To get a better idea of what works – and what doesn’t – for your audience, take some time to comb through your recent campaigns.

Which sends garnered higher open rates than usual? Which didn’t make much of an impact? And what do their subject lines have in common?

Start with this information, then implement A/B tests to confirm your findings. 

Add an emoji (or two)

From my research, I’ve found a few common tactics that consistently boost open rates. And, yes, emojis are first on the list.

• Emojis: I’ve rarely seen a subject line sans emoji outperform a subject line with an emoji. To make a bigger impact, try using unique, non-yellow emojis and place them at the beginning or in the middle of your subject line.

• Non-irritating grammar: Customers typically respond better to copy that doesn’t annoy them. Try using statements instead of questions (especially if that question can be answered with a “no”), keeping exclamation points to a minimum, and capitalizing one word at most.

• Specifics: Don’t be coy. Tell your audience exactly what they need to know by clearly stating the details of your promotion, exactly when it ends, and mentioning products by name if applicable. Simultaneously, it’s still important to entice the customer to open the email by alluding to more details or building urgency. 

Of course, these are not guaranteed to improve performance for every brand, but my findings indicate the impact will likely be positive.

Give these basics a shot to see which best resonates with your recipients.

Follow best practices

An A/B test should be as simple as that — A and B.

No need to overthink it. Choose a single, straightforward element you’d like to test and isolate that variable. If there is more than one difference between your A subject line and B subject line, there’s no way to clearly indicate what affected the results.

For example, take a look at this A/B test:

SUBJECT A: 🚨😱 20% OFF ends today 😱🚨

SUBJECT B: 20% OFF ends today 😳

If subject A receives a higher open rate, we don’t know what to attribute that win to. Perhaps the specific emojis, maybe their placement, or possibly that there are four.

Whereas, with this test, the results are much clearer.

SUBJECT A: 🚨 20% OFF ends today

SUBJECT B: 😱 20% OFF ends today

If subject A outperforms subject B, it proves that the alarm emoji was more effective – in this instance anyway.

The results of a single A/B test do not determine anything with certainty. So, continue testing and retesting to solidify your findings.

Set up your A/B test last

Klaviyo makes A/B testing subject lines very quick and easy. 

Simply navigate to the content portion of the editor and click “Create A/B Test.”

This will duplicate your email. Therefore, it’s imperative to ensure the email content is complete and doesn’t require any further edits. If you do make a change, you will need to do so in both versions of the email. 

From there, enter your A and B subject lines in their respective fields. Then, select “open rate” as the winning metric and choose your test size and duration.

A 50/50 test size is meant solely to learn from your results, but I recommend testing with a smaller sample size of your audience. For example, if you’re sending an email to a list of 10,000 recipients, perform your A/B test with a sample size of 10%. Klaviyo will send the version that performs best to the remaining 90%.

By doing this, you’re both gathering valuable data and applying it within the same send.

To set your A/B test in motion, simply schedule your email send as you usually would.

Analyze the data

While you’ll absolutely notice trends from quickly checking the results of an A/B test, you’ll get a much better picture by carefully scrutinizing the details of each send.

Record your results

Wait at least 48 hours from the time you send a campaign before collecting any data. This gives the email time to reach your audience, providing more accurate results.

When you’re ready to start logging, create a running spreadsheet with the results of all your A/B tests as you perform them.

You should have a column for each of the following:

• Date of send
• Campaign name
• Subject A
• Subject A open rate
• Subject B
• Subject B open rate
• Testing variable
• Notes

In that last column, you can jot down your findings, thoughts, and ideas for future tests. 

Depending on the volume of emails you’re sending and other results you’re interested in assessing, you might consider adding columns for send time, click rate, conversion rate, and other data points.

Understand statistical significance

Not every A/B test provides compelling results that forever impact the way you write subject lines.

Klaviyo automatically marks each A/B test as one of four categories of statistical significance:

• Statistically significant
• Promising
• Not statistically significant
• Inconclusive

If your A/B test results in the latter two, this means the difference in open rate was very small (less than ~0.5%) or there was not enough send data available, respectively. You likely can’t glean much information from these results, but you can always try testing those variables again.

However, a statistically significant or promising A/B test result means at least 50 people received each variation of the email and Klaviyo predicts a win probability of >90% or >75%, respectively.

These findings hold more weight. Even if the percentage difference between the two open rates is just 1%, that miniscule-seeming number could represent hundreds or thousands of customers!

Consider other factors

A/B tests don’t exist in a vacuum, so it’s imperative to note potential factors that may have impacted open rates of both subject lines and the percentage difference between them.

For instance, when promoting a sale or new product launch, open rates are likely to be above average.

However, even a single variable change between your A and B subject lines can make a more drastic difference than you might imagine.

SUBJECT A: This is our best seller 

SUBJECT B: The Cinnamon Pumpkin Candle is our best seller 

Here’s an example testing the vague article (“this”) against mentioning the product by name. But this causes another distinction: length. Therefore, if subject A wins, that may be attributed to mystery piquing the reader’s curiosity or the fact that it’s shorter and more concise than subject B.

How to apply the results for BFCM

Now that you’ve tested a variety of variables and discovered what your audience responds best to, you can use those findings to write strong, eye-catching subject lines.

Implement multiple proven variables

Let’s say that after a month of A/B testing, it’s clear that your customers are almost always more likely to open emails with subject lines that include an emoji, the explicit mention of a discount, and one, short capitalized word.

These elements are proven to boost open rates on their own, so you should aim to employ at least one (if not all) in every single BFCM email subject line. For example:

Just be careful to avoid repetition, which could get on your customers’ nerves.

Consider keeping track of the subjects and previews for every email in your Cyber Week strategy in one place. This gives you a high-level overview where you can compare for consistency and ensure each subject line is unique.

Here are three similar subjects that use all three proven variables, but still offer variety.

SUBJECT: 20% OFF starts today 🤩

SUBJECT: 20% OFF 🚨 Our biggest sale of the year 

SUBJECT: ⏰ 20% OFF ends at midnight

Don’t A/B test during Cyber Week

Can you keep up the testing throughout BFCM? Sure. Should you? I wouldn’t recommend it.

This is likely your business’s busiest and most financially lucrative period of the year. There’s no need to create even more work for yourself or your team.

Instead, focus on ensuring each email contains the correct information, is scheduled to go out at the right time, and is set to send to the right list or lists. This is incredibly important and deserves as much of your attention as possible.

Plus, there’s still plenty of valuable data you can collect from your results without A/B tests.

Analyze the data (again)

Once Cyber Week wraps up and you’ve had a chance to catch your breath, take some time to assess how well your BFCM campaigns performed.

Consider what the emails with the highest open rates have in common. And do the same for those with lower open rates.

Let’s take another look at these subject lines and their resulting open rates:

SUBJECT: 20% off starts TODAY 🤩 (48%)

SUBJECT: 20% OFF 🚨 Our biggest sale of the year (56%)

SUBJECT: ⏰ 20% off ENDS at midnight (49%)

From this very small and hypothetical data set, we can learn a lot. The second subject line had the highest open rate by far. This could be due to “off” being capitalized, the siren emoji, or its position in the middle of the subject line.

These are all observations worth noting and further A/B testing in the future.

What about Apple Mail Privacy?

While Apple's Mail Privacy Protection (MPP) has altered the landscape of email marketing as it pertains to open rate accuracy and metric analysis, it doesn't mean open rates should be totally ignored. 

Open rates are inflated due to MPP, but if you are running enough subject line a/b tests across the same audience, you can gain valuable insights just as you would have in pre-MPP times.

Key takeaways

• Start A/B testing now. Like, immediately. Setting up an A/B test is quick, easy, and incredibly valuable. Don’t waste any more time and gather as much data as possible before November rolls around.
• Test and test again. Never take the results of a single A/B test as gospel. Continue testing to solidify your findings and ensure you’re writing the strongest subject lines possible.
• Isolate one variable in your A/B tests. This makes writing two subject lines so much simpler and gives you clear, easy-to-read results.

Want to offload your email work? 

Our team has designed and written emails for BK Beauty, Natural Dog Company, American Tall, and more. You can learn more about our email services here.

Click here to take our quick quiz (1 minute tops) and we'll send you a free Cyber Week roadmap!