The multi-armed bandit test is one of the primary methods that growth teams deploy when they set up an experiment. Below, I will make this complicated method seem simple, and I will describe some inspiring case studies.
In their toolkits, growth teams have several methods to experiment. One of the more complex “flavors” of testing is the multi-armed bandit. Some readers of this blog (shout-out to Christina and Henk) told me that it is hard to find an easy-to-understand explanation of this method. This article should fill in the blanks.
Some Background: A/B Testing
Before I delve into the nitty-gritty of multi-armed bandit testing, I will take a short detour. Because multi-armed bandit tests are more complex versions of A/B tests, first you have to fully understand what A/B testing is.
Also known as a “split-run test,” an A/B test is a controlled experiment in which two versions of a single variable are compared by testing a subject’s response to variable A against variable B. This will determine which of the two variables is more effective.
This method is one of the keys to the success story of Netflix. Almost all their decisions are based on data analytics and A/B testing. Through split-run testing, Netflix selects the best artwork for its content, sometimes resulting in 20% to 30% more viewing for that title.
Growth hackers can do the same. A simple example: through split-runs, they test the effectiveness of your price by showing different rates to different groups of website visitors (segmented in customer type). For each group, a rate is calculated which ensures the highest profit. Each group will now see the price that is calculated for them. Variations in prices based on time of day are also possible.
So What Are Multi-armed Bandit Tests?
Multi-armed bandit tests are “smarter,” more complex versions of A/B tests. How does it work? Machine algorithms are used to dynamically allocate traffic to variations of a website that perform well while assigning less traffic to variants that perform worse.
These tests work faster than A/B tests because they send traffic to “winning” alternatives to a greater extent within a specified period. In this way, less time is wasted on testing a poorly performing option (as happens with A/B tests).
Multi-armed Bandit Test Examples
Multi-armed bandit algorithms are not only used in testing, but also daily applications. Take for example these inspiring cases:
- The famous personal recommendations of Netflix are based on a multi-armed bandit algorithm.
- The website of The Washington Post shows combinations of articles with the largest chance that you will click on them.
- Wireless networks use a multi-armed bandit algorithm to determine which route is optimal and the most energy-efficient.
You may want to know when you will meet a multi-armed bandit in your business. Well, because of their speed, these tests are particularly suitable for situations where rapid results are required, for example, variations on news headlines. Your growth team will possibly deploy this method to test short-time campaigns.
A Final Word
Growth hacking can mean the difference between the businesses that fail, survive, or thrive. Through our experienced partners and staff, RevelX combines the wisdom of more than 100 years of growth hacking. If you want to know more about this inspiring subject, don’t hesitate to reach out to me. You are always welcome for a cup of coffee at our office.
Björn Brekel
Lead growth hacker @ RevelX
Bringing startup thinking to the boardroom
Road cycling, minimalism and blockchain enthusiast
Related Posts
August 13, 2020
5 Growth Mindset Examples That You Can Use in Your Job Tomorrow
People who believe their success is…
April 22, 2020
Why customer lifetime value analysis is now more important than ever
Over the course of the last weeks we,…
March 13, 2019
7 Cognitive Biases and How to Use them in Marketing & Sales
Consumer behavior is shaped by the same…
March 4, 2019
9 Psychological Tricks to Influence Consumer Behavior (and Grow Your Business)
As marketers and salespeople, you…
February 6, 2019
What Is Reviewification, and Why Is It Changing Consumer Behavior?
Customer reviews are all around us and…
January 25, 2019
The Readability Rulebook: How to Write More Effective Copy
Your online copy's readability is…
December 17, 2018
Want to Optimize the Customer Journey? Try These Tools!
The concept of the "customer journey"…
December 10, 2018
How I Used a Go-To-Market Plan to Innovate Home Decoration
This is an inspiring example of a…
October 1, 2018
Why You Need a Go-To-Market Plan (and How to Create It)
Just launching a product can result in…
July 30, 2018
How Augmented Reality Improves the Shopping Experience
Augmented Reality is changing the way…
July 23, 2018
Before You Start Growth Hacking…Fix the Plumbing!
Before you start growth hacking, there…
July 17, 2018
3 Ways External Data Can Help Grow Your Business
We’ve all heard that we should be using…
June 29, 2018
7 Tools that Offer Direct Scheduling to Instagram
For years, scheduling Instagram posts…
June 13, 2018
Vanity Metrics vs. Actionable Metrics: Measure What Matters
Measuring and analyzing data is one of…
June 1, 2018
How Amazon, Google, and Belsimpel Perfected the Customer Journey (And How To Apply These Lessons)
Some companies confuse the different…
May 9, 2018
Fail Hacks: 6 Amazing Tricks to Kill Your Conversion Rates
Read these amazing “fail hacks” that…
May 7, 2018
Digital Growth Hacker Marketing: The Ultimate List of Amazing Case Studies
24 inspiring examples of growth-hacker…
April 30, 2018
Artificial Intelligence Is Not the Answer To All Marketing Challenges (and That’s OK)
The possibilities of artificial…
April 24, 2018
4 Amazing Tips to Optimize Your Landing Page and Generate More Leads
To generate a maximum number of leads,…
April 13, 2018
Checklist: 3 things to cover privacy issues when using website conversion pixels
Setting up conversion pixels are useful…
April 9, 2018
5 Reasons Why You Need Content to Enhance the Customer Experience
Excellent content is a must for every…
March 20, 2018
3 Examples of Failed Commercial Performance (and What to Learn From Them)
The chain is no stronger than the…
March 14, 2018
Low-hanging Fruit: 9 Growth Hacks You can use TODAY
For real growth, a long-term strategy…
March 12, 2018
What I Have Learned From YouTube: How to become more customer centric
YouTube is one of the biggest successes…
September 11, 2017
Growth and Onboarding: the 3 best tips for successful user onboarding
10 common misconceptions on writing…
October 20, 2016
7 Neuromarketing Basics to ‘Hook’ New Customers
In the past, neuromarketing was limited…
September 26, 2016
10 irrational tips for writing winning proposals
10 common misconceptions on writing…