Sequential price testing in e-commerce refers to a process where experiments with prices are conducted sequentially over time. The merchant doesn’t offer different prices for a product at a same time. This means that sequential price testing prevents from price discrimination and allows you to iteratively adjust and learn over time while being fair with your customers.
One effective method to optimize pricing in a structured yet efficient way and running sequential tests is the multi-armed bandit (MAB) framework. Unlike traditional A/B testing that splits traffic evenly between options, the MAB framework dynamically allocates more traffic to the most promising prices as it gathers data. This results in faster and more efficient price discovery with less revenue loss.
In this article, we’ll explore how we apply MAB framework to sequential price testing for different product categories, giving practical examples of testing two, three, and five price points.
What Is the Multi-Armed Bandit Framework?
The MAB framework is a type of reinforcement learning used for decision-making problems with multiple options. In the context of price testing the goal is to find the price that maximizes a predefined reward (e.g., conversion rate, revenue, profit).
The framework dynamically adjusts how much traffic is allocated to each price based on the real-time performance. As more data is collected, the model increases the traffic towards the price that performs the best, speeding up the learning process.
How Sequential Price Testing Works?
- Define Price Variations: Choose multiple price points to test for a given product.
- Define objective: Consider revenue, profit or conversion rate optimization. In case you want to optimize the profit from individual products, you need to provide the cost of the product to the model so that it will optimize your margin. In order to learn, the models reward sales and panelize page views without purchases.
- Choose pricing level: You can perform product level or variant level pricing. In case of product level price testing, your products will have identical price no matter: color or size for example.
- Apply business rules: Insert min margin percentage or min-max price range to be sure the model will not go below or above limits.
- Define price change frequency: For a period of time a particular price will be visible to all customers then change to another and so on. Heavy traffic websites can change prices every hour, while others can change once per day or even slower once per week. The frequency determine how many price testing periods you will have within a particular campaign.
- Convergence: The algorithm converges on the best-performing price, gradually minimizing the traffic sent to underperforming price points.
Now, let’s explore examples of how the MAB framework can be used for sequential price testing with different product categories.
Dresses (Two Price Points)
Product: Women’s Floral Dress
In this case, we’ll test two price points for a summer floral dress to find which price maximizes both sales and profit.
- Price 1: $50
- Price 2: $60
The MAB algorithm will initially send 50% of traffic to each price point. As data accumulates, it might observe that the conversion rate is higher for the $50 price, but the overall profit is greater at $60 due to a higher margin. The algorithm will reallocate more traffic to the $60 price as it learns which is optimal for balancing revenue and sales volume.
Results:
- Higher Conversions: The $50 price attracts more buyers but results in less revenue per sale.
- Higher Revenue per Sale: The $60 price converts fewer customers but leads to higher revenue.
Over time, the MAB may converge on $60 if the total revenue generated is higher, or it may settle on a mix if both prices perform similarly in terms of revenue.