Testing and Minimum Order Sizes – What is Best Practice?
by Stephen Lett
When it comes to catalog marketing, I don’t like to leave anything to chance. Just about everything can (and should) be tested including promotional offers, cover designs, minimum order requirements, etc. Knowing what and how to test and re-test is important to the success of any catalog. There are a few basic rules to follow when testing which I want to review this month. I also want to focus on a specific test having to do with setting order minimums to qualify for offers and how you might go about setting up a test of your own.
I see minimums to qualify for promotional offers being set too high. Instead of encouraging people to order, order minimums can actually have the opposite effect. If your average order size is $65 it doesn’t mean that 50% of the orders are above the average while 50% are below. The typical distribution of orders by dollar ranges would be as follows:
| ORDERSBYDOLLARS.xls LETT Direct, Inc. Date: 12-15-07 |
% ORDERS BY DOLLAR RANGES | |
|---|---|---|
| ORDERS | DOLLARS | |
| DOLLAR RANGES: | ||
| $0.00 to $65.00 | 70.00% | 37.00% |
| $0.00 to 100.00 | 87.00% | 59.00% |
| $0.00 to $150.00 | 96.00% | 76.00% |
| $0.00 to $200.00 | 96.00% | 80.00% |
| $0.00 to $250.00 | 98.00% | 89.00% |
| $0.00 to $500.00 | 99.90% | 94.00% |
| $500.00 and More | 100.00% | 100.00% |
This chart shows that 70% of the orders (37% of the dollars) fall below the average order size. Therefore, if you are offering free shipping on all orders above $99 most orders fall considerably below this amount. It is too much of a stretch for someone to reach the minimum. It would still be a reach to set the minimum order at $70 just slightly about the average order size in our example.
When you consider the percentage of orders that fall below the promo order minimum that you would typically set, I think it makes a great deal of sense to consider testing no minimum. I know this is a scary thought to you but test after test has supported making offers without minimums (or at least setting minimums much lower than your typical average order). Based on the testing I’ve done, you should see a significant increase in the RPC (Revenue per Catalog); of course, your actual results will most likely vary. Both the response rate and average order size will most likely be higher with no dollar order minimum. Most of the benefit comes from an increase in the response rate since more people are able to qualify for the offer. This is a good thing because it means more people are ordering and being added to your 12-month buyer file. Shown below is a summary of actual test results for no minimum vs. a $99 minimum vs. the control, i.e., no offer.
| OFFERTESTRESULTS.xls LETT Direct, Inc. Date: 12-03-07 |
FREE SHIPPING OFFER NO MINIMUM vs. $99 MINIMUM vs. CONTROL |
|||
|---|---|---|---|---|
| AVG. | RESPONSE | |||
| SEGMENT | ORDER | RATE | RPC | LIFT |
| Buyer Housefile: | ||||
| Free Shipping - No Order Minimum | $65.39 | 2.68% | $3.23 | 33.45% |
| Free Shipping - $99 Order Minimum | $71.95 | 2.21% | $2.93 | 21.09% |
| Control - No Offer | $68.75 | 1.91% | $2.42 | ------- |
| Cooperative Databases: | ||||
| Free Shipping - No Order Minimum | $59.25 | 1.05% | $1.20 | 33.28% |
| Free Shipping - $99 Order Minimum | $57.91 | 0.90% | $1.00 | 11.66% |
| Control - No Offer | $57.63 | 0.81% | $0.90 | ------- |
| Outside Rented Lists: | ||||
| Free Shipping - No Order Minimum | $66.40 | 1.07% | $1.30 | 43.11% |
| Free Shipping - $99 Order Minimum | $61.29 | 0.95% | $1.06 | 17.28% |
| Control - No Offer | $58.41 | 0.85% | $0.91 | ------- |
The percent lift is calculated from the control, i.e., no offer. For example, to the housefile, free shipping with no order minimum increased the RPC 33.5% over No offer. Free shipping with a $99 order minimum increased the RPC 21.1% over the control. Also notice how much better the “no minimum” results were compared with the $99 minimum segment.
With regard to prospects, I often see an increase in both the response rate and average order size. When there is a minimum, consumers try to get to the minimum and once they do, they stop buying. With no minimum, they tend to shop/spend more.
Test an offer with no minimum against the control, i.e., normal minimum order size. Or, this could be a three way test, i.e., the offer (no minimum) vs. the offer with a minimum vs. no offer at all. This should be tested to both the housefile and to prospects. It is important to execute the test properly so you can read the results. When testing you should follow “best practices” guidelines. I have found over the years that following these simple rules insures an accurate, measurable result and a sound conclusion.
Simple Rules For Testing
- Clearly define the purpose of the test; define the objective.
- Prepare a pro-forma; do your financial analysis.
- Always test against a control.
- Only test one variable at a time; keep everything else the same.
- Don’t test during your peak season (unless you need to).
- Always re-test against a control or another offer.
- Make sure your sample size will yield statistically valid results.
- Source code the control group and test groups properly.
- Read the results and act on what you see!
Structuring a test properly is essential for actually measuring the results. Focus on keeping all elements such as the creative the same except for the one variable you’re testing or changing. And, always test against a control. The difference between a control group and a test group is what you’ve been doing. For example, if you have not been making any offer, this is the control. If you always offer free shipping, this becomes your control. Doing something different from the control, for instance, offering a promotional incentive to a group of customers or prospects would be the test group. The same catalog should be mailed to all test panels on the same mail date. If you’re promoting the offer on the front cover for one test panel, do the same for the other test panel. Promoting your offer on the cover for one group and on the inside order form for another group doesn’t constitute a valid test.
Sample Sizes
I always feel it takes a minimum of 100 orders from any one group to accurately “read” the results. (Sometimes less depending on leap you are willing to make when rolling out.) For example, a minimum of 100 orders per panel at a 1% response rate to prospects would mean your panels need to be 10,000 each. If you are testing two promotional offers against a control, i.e., no offer, you need to print 10,000 copies for the “A” group (the control group) and 10,000 each for the “B” and “C” groups (the test groups). Then, assign key codes to all three panels. This will enable you to track the results. Select a representative sample across all zip codes. All three groups need to be selected from the same list universe. They all need to be mailed on the same day. Once the mailing is at least 50% complete you can predict the winner with confidence.
Knowing how to test is critical to your success. This is the only way you can determine what worked (and didn’t work). Knowing what to test is critical too. Most things can be tested. Don’t assume you know what will work or not work. Test. Test. Test! The results might even surprise you!