How to A/B Test Shopify Emails

We needed to a/b test a banner in the “Order Confirmation” email that Shopify sends. Here’s how we did it:

First, at the top of the email template in Shopify, we created a variable called, “randomNumber” using a random number generator, which outputs either a 1 or 2 (i.e. “true” or “false”):

{% assign min = 1 %}
{% assign max = 3 %}
{% assign diff = max | minus: min %}
{% assign randomNumber = "now" | date: "%N" | modulo: diff | plus: min %}

Thanks to https://www.131-studio.com/blogs/shopify-conversion/generate-random-numbers-using-liquid-shopify

Then, in the body of the email template, we added the following if…else statement:

{% if randomNumber == 1 %}
       <p>Learn more at <a href="https://url.com?utm_source=email&utm_medium=shopify_email&utm_campaign=order_confirmation&utm_content=control">Our Store Name</a></p>
  {% else %}
      <p>Learn more at <a href="https://url.com?utm_source=email&utm_medium=shopify_email&utm_campaign=order_confirmation&utm_content=variation">Our Store Name</a></p>
{% endif %}

Note the “control”/”variation” text in the “utm_content” url parameter, which will allow us to filter the results in Google Analytics.

That’s it! Let me know if you’ve found a different way to run an A/B test in a Shopify transactional email.