Hold Out Testing: To Mail, or Not to Mail?

By: Sandy Wolstencroft

 

To Mail, or Not to Mail?  Is it better to mail less frequently, but deeper?  Or, are more touches to your core buyer file the best approach for your print contact strategy? 

Before there was digital influence in the path to purchase, these were very easy questions to answer, segments which performed above breakeven were mailed.  Period.  Today, there are so many additional touch points we have with our customers, these questions are not as simple to answer.  For companies that have active print and digital programs, we have found that roughly 30-50% of the sales tracked to a mailed segment would have come in without the influence of the mailing.  On the other hand, the sales activity coming from the other 50-70% influenced by the mailing is significant and essential to the health of the overall business. 

One way to test this is for your business is to create hold out testing.  Select segments will be split.  One half of a segment will receive mailings and all other touch points, and the other half will not be mailed but will receive all other touch points.  To read the results, capture all sales activity for the period, and calculate performance rates and contribution for each test panel. 

Over time (and a lot of testing), we have found that mailing your Top Tier/Core buyers results in a higher level of orders, sales and profit vs. not mailing Top Tier/Core buyers.  The same holds true for your Mid Tier buyers.  Below is a recap from a recent Hold Out test we conducted:

*Profit is calculated after allowing for COGs, Returns/Cancellations and Cost in the mail.

The results above show that Top and Mid Tier buyer segments should be mailed.  The increase in Orders, Demand and Profit all lead to a stronger 12 month buyer file, and contribute to your bottom line.  The Low Tier buyer segments brought in more Orders and Demand, but lower Profit.  In this case, we would suggest that you use modeling to select the best subset of this buyer group, include a promotion for this buyer group to improve results and/or continue to mail this group, but less frequently.  This is where you need to decide if the time is right for you to invest in building your 12 month buyer file, or be more conservative with your ad dollars.

Another test we would like to share with you was designed to confirm that mailing more frequently to your Top and Mid Tier buyers is a sound contact strategy.  In this case, we tested Monthly vs. Bi Monthly mailings. 

Orders, Demand and Profit were higher when Top Tier and Mid Tier Buyers were mailed more frequently.  In this case, the Low Tier buyers (not shown) performed at roughly $1.50/Book when mailed during the period of this test.  This is significantly lower than the results from mailing Top and Mid Tier more frequently.  The conclusion is that mailing more frequently to Top and Mid Tier buyer segments is a better strategy vs. mailing deeper and less frequently. 

Additional information to note:

These results will vary, which is why we always recommend testing.  However, please note that there is risk of hold out testing your Top Tier buyers, which can be pretty high.  Before moving forward with a test, do a quick calculation to determine how much demand dollars are at stake by not mailing a portion of your buyer file. 

On the other hand, if you are not already mailing to your buyers, the risk of not testing is high.  There is an enormous opportunity to increase orders, demand and profitability by sending mailings to your buyer file and testing to prove that potential is recommended. 

There is great value in keeping your housefile actively buying, even beyond what these figures show.  Having your mail piece in the hands of your customers helps keep your brand in your customers’ mind, rather than on your competition.  Every touch point, and every order is an opportunity to strengthen the bond between you and your customers. 

Sandy Wolstencroft has over 30 years of catalog and direct mail experience. She was the Marketing director for mid-large sized catalog companies before joining Lett Direct as a Marketing Consultant 20 years ago.  Today, she heads up the print division of Lett Direct.  Sandy also has service bureau experience, which gives her special insight into customer data.  She strongly believes that there is a numbers driven story behind all results, that supports a clear marketing path.  She can be reached at sandyw@lettdirect.com