How to A/B Test Shopify Emails

We needed to a/b test a banner in the “Order Confirmation” email that Shopify sends. Here’s how we did it:

First, at the top of the email template in Shopify, we created a variable called, “randomNumber” using a random number generator, which outputs either a 1 or 2 (i.e. “true” or “false”):

{% assign min = 1 %}
{% assign max = 3 %}
{% assign diff = max | minus: min %}
{% assign randomNumber = "now" | date: "%N" | modulo: diff | plus: min %}

Thanks to https://www.131-studio.com/blogs/shopify-conversion/generate-random-numbers-using-liquid-shopify

Then, in the body of the email template, we added the following if…else statement:

{% if randomNumber == 1 %}
       <p>Learn more at <a href="https://url.com?utm_source=email&utm_medium=shopify_email&utm_campaign=order_confirmation&utm_content=control">Our Store Name</a></p>
  {% else %}
      <p>Learn more at <a href="https://url.com?utm_source=email&utm_medium=shopify_email&utm_campaign=order_confirmation&utm_content=variation">Our Store Name</a></p>
{% endif %}

Note the “control”/”variation” text in the “utm_content” url parameter, which will allow us to filter the results in Google Analytics.

That’s it! Let me know if you’ve found a different way to run an A/B test in a Shopify transactional email.

Posted in ,

Toby Cryns

RSS From Toby’s Blog

  • The Horrors of Covering Your Own Ass
    Today I emailed a very simple question to support@[companyname].com (2 sentences total). The automated response I got back was this: Keep in mind that the above was an auto-response and: It’s hard to read. Everything has the same import (e.g. none of it is important enough to call out separately). It’s long. It added absolutely-zero…
  • If something stupid makes money, then it’s not stupid.
    I’ve seen some stupid things in my life…And some of them were really really smart. I remember a story from decades ago where the U.S. military was looking to equip the Stealth Bomber with a fancy computer-driven video system so the pilots could see behind them. Well, the computers failed to get the job done,…
  • There is no destination.
    Coding is a process of failure followed by a moment of great joy. You've gotta ride through the failures to experience the joy...

More posts from themightymo.com

How to deactivate all WordPress plugins via the database

By The Mighty Mo! Design Co. | January 20, 2022

Go to phpMyAdmin. Go to the “wp_options” table. Search for the field named, “active_plugins”. Edit the “active_plugins” field. Delete the value/contents of the “active_plugins” row. Save. That’s it! All your plugins are now deactivated!

Website DNS, Domain Registration, & Hosting Basics

By Toby Cryns | November 30, 2021

Websites have multiple layers: Domain Registration When you register a domain, you pay someone ~$20/year for a .com domain. Registration means you are leasing the domain for a year or more. A domain is the “yoursite.com” or “yoururl.net” or “yourorganization.org” that people type into their browser. We use services like Namecheap, Dreamhost, and GoDaddy for…

Avada ThemeFusion loads 80+ javascript files on every page! DON’T RECOMMEND!

By The Mighty Mo! Design Co. | November 18, 2021

I inherited a WordPress site using Avada ThemeFusion, and it is experiencing really bad performance issues. After doing lots of my standard bag-of-tricks optimizations to little effect, I decided to simply count the javascript files. To my surprise horror, I found it was loading over 80 javascript files on every single page! (For comparison: themightymo.com…

RSS From Toby’s Blog

  • The Horrors of Covering Your Own Ass
    Today I emailed a very simple question to support@[companyname].com (2 sentences total). The automated response I got back was this: Keep in mind that the above was an auto-response and: It’s hard to read. Everything has the same import (e.g. none of it is important enough to call out separately). It’s long. It added absolutely-zero…
  • If something stupid makes money, then it’s not stupid.
    I’ve seen some stupid things in my life…And some of them were really really smart. I remember a story from decades ago where the U.S. military was looking to equip the Stealth Bomber with a fancy computer-driven video system so the pilots could see behind them. Well, the computers failed to get the job done,…
  • There is no destination.
    Coding is a process of failure followed by a moment of great joy. You've gotta ride through the failures to experience the joy...