Digital marketers have a number of advantages over their Mad Men era predecessors. Better analytics, more touch points, increased engagement with consumers, the list goes on. But the biggest advantage we have is that we can test and optimize campaigns.
Once a billboard is up, it’s difficult and expensive to change it if people don’t like it. When it comes to digital however, you can optimize endlessly. With testing you can:
- Discover which email subject line generates the most opens
- Find the most compelling copy for a push notification
- Tweak elements on your website to increase conversion
And that’s barely scratching the surface. Google is famous for testing everything from the number of search results to display on a page to the shade of blue on a particular button.
But before you can utilize testing effectively, you need to understand the different types, and that can be difficult. A/B Tests. Multivariate Tests. Machine learning. What’s the difference? Let’s take a look at each and the advantages and disadvantages they offer.
A/B testing is a simple experiment contrasting the performance of two variants (variant A, and variant B. A/B testing!) A small number of messages are sent to each test group. Whichever variant has higher goal achievement is the winner and is used for all of the remaining messages.
Example: Comparing the copy of two separate push notifications to see which performs better.
- Results are easy to understand
- Relatively simple to design and execute
- Takes a long time to test
- Can only test two variants
By contrast, multivariate tests allow you to test multiple variables and variants at the same time. Just like A/B tests, a small number of messages are sent to each test group. Whichever variant has the highest goal achievement is the winner and is used for all of the remaining messages.
Example: Comparing the copy, time sent, and emoji used of separate push notifications to see which performs better.
A relatively new addition to the optimization game is machine learning. Machine learning takes the logic behind A/B tests and multivariate tests and outsources the number crunching to a computer. Messages are sent on a continual basis, and the computer scales up the send volume of the variants that are performing well, and scales down the under performers.
Example: Comparing the copy of 4 separate push notifications to see which performs best.
- Optimizes in real-time, which makes it significantly faster
- Optimization happens automatically, with no input from marketers required
- Far more accurate than other tests
- Provides detailed insight on which variables perform best
- Requires the right tool, like Kahuna!
Want to learn even more about what the right mobile tool can do for you? Download our free Mobile Marketing Automation Guide today!