Response Analysis Made Simple

The pressures of promoting and fulfilling peak-season sales can overwhelm the subtleties of tracking exactly where those sales are coming from. If you have a real-time tracking system or already analyzed your holiday results, you should have a good idea. If you don’t and you didn’t, now is the time to set up your analysis to inform you of what happened and to prepare you for when you’re in the thick of the holidays again.

Where do you start?

Before you can combine your data by segment, list, and mailing as well as your Web data, you must find your files and put them into a common format, then set up your analysis framework. Pull together all the prospecting lists and house file segments you mailed and e-mailed. Add the quantities mailed for each catalog or offer, the keycodes you assigned, and the promotional costs of printing and postage.

If you had forecast results by segment based on the previous year’s history, be sure to add those projections to your master list. The time you spent forecasting will pay off only if you compare your figures with your customers’ and prospects’ quantified behavior.

This master list of what you mailed is the framework to which you can attach all your response data: numbers of orders, costs of goods sold, gross sales, returns, and discounts. If your online offers were simply electronic versions of your catalogs, include those results with your hard copy mailings’ sales. If your Web sales came through distinctly different media — e-mailed offers to opt-in lists, or a comprehensive Website presentation of all your products not linked to your mailed offer — evaluate those results separately.

If you have orders that came in without source codes, try matching them to your master list by name and address. Try using address alone if you suspect your mailing might have been passed along within a household or an office.

As a last resort, if you still have a large number of unassigned orders, allocate them across all your prospecting promotions, using the response that you can measure to weight the distribution accordingly. For instance, if you have 100 unassigned orders and two lists to distribute them between, don’t simply assign half of the orders to each list. Look at the response rates among the orders you were able to allocate; if list A had twice the response rate of list B, assign 67 of the unmatched orders to list A and the remaining 33 to list B. That way you will give credit to your better efforts and account for the sales momentum gained through your total efforts.

What do you look at?

Once you have matched your response data of orders and sales received to your lists and segments, you should look at four standard measurements:

  • Response percentage by list and segment — dividing orders from each offer by the quantities mailed of each segment and each offer.

  • Average order size — total sales by segment divided by total orders.

  • Response per promotion or catalog — total orders keyed to specific mailings divided by the quantities mailed.

  • Return on investment — gross sales minus discounts and returns divided by the list rental and mailing costs. This can also be figured on profit dollars per dollar of promotional costs.

What can you conclude?

You can see as you look over your results by mailing and by mailed segment and list both how accurate your forecasting was and the figures on which to base your future forecasts and plans. The performance of your mailings can be evaluated by sorting and scoring your results in three ways using the four measurements above.

  • Sum up each house segment’s sales and orders for each of your mailings. Sort the segments from high to low based on number of orders, average order size, response percentage, and return on investment. Then sum up your house segments across the season so that you have each segment’s sales, orders, average order size, and response percentage across all mailings from October to December. Sort your house segments and fill in their ranks from high to low for average order size, percent of households or individuals responding against all the mail pieces you sent them, and total orders. These rankings, within each individual mailing and across all your holiday mailings, should correspond to your preseason or prior year’s recency/frequency/monetary value (RFM) scores. All of these households are recent, so their frequency and monetary values will be critical in comparing results with projections and should guide you in adjusting your RFM scoring process if necessary.

  • Sum up each prospecting list’s sales and orders for each mailing. Follow the same procedure used in determining the sales and number of orders for house file segments. Calculate the results for each list, and sort and rank the lists for each mailing in which you used them. Then add up all their orders and sales from all holiday mailings, and sort them from high to low. Finish by assigning ranks to evaluate them as sources of new customers for the whole season.

    This will tell you which lists worked and which failed. Note if particular families of lists succeeded or not. Lists from a single list manager that didn’t work during the holidays might not deserve another chance the rest of the year. You can then go deeper into the sources of the lists that did work and test the high-ranking lists for nonholiday use as well as to include them on your list of files to use for the holiday season.

  • Add all your house segments’ orders and sales together for each of your catalogs and specific promotions. Sort and rank the individual mailings — as you had the house file segments and the prospecting lists — to see how each compares with the others as a moneymaking effort. Analyze prospect lists separately, totaling orders and sales for each mailing, and then sorting and ranking the mailings as sources of new customers and new sales. Compare these figures to your prior year’s results on a last-year-to-this-year basis.

What do you know?

These views of your holiday results will tell you which of your customers bought from you and which gambles on rental lists paid off. They will show how each mailing and how your total marketing effort paid off, as well as provide you with a complete, detailed template for holiday 2007.


Bill Singleton is president of Singleton Marketing, an Algonquin, IL-based consultancy specializing in database marketing.