Many marketers think that by randomly allocating multibuyers they get a statistically valid read on their overall list effectiveness.
Not so, and here’s why: The merge conceals true list performance, which is what’s needed to make informed decisions regarding your list sources.
That’s because this technique, which I call “merge skew,” represents only post-merge results. They are typically biased because a merge isn’t completely random if it has levels. Any ordering, even done randomly within groups, can skew results.
But let’s say merges were in fact done at random. Would they provide complete insight into true results? Not really, because the decreased sample size would create noisier estimates of that list’s performance. In fact, any results read from only a subset skews results by the biases in the merge. Therefore, the accuracy of the estimate regarding that list’s true performance suffers.
An example would be that if you had a List A that held 10,000 names taken in priority and that performed at 1%. Meanwhile, you have a list in lower priority – List B – that has 7,000 names and a post-merge performance of 0.7%. But let’s say List B has an overlap of 5,000 names with List A, and that those 5,000 names are performing at 2%. Now you’d have conflicting data as well as useful data that would be difficult to discover – basically, a noisy estimate.
Since it’s possible to get true pre-merge results with very little – if any – incremental cost, doesn’t it make sense to do what it takes to get those results?
Have your service bureau take all the mailed names from each list source and allocate back the responders regardless of whether that list source got in-the-mail credit for the name or not. This way you won’t give a list source credit for sub sections of your performance; you’ll give the source true credit for the full value of the names provided.
Now that you have clarity around list source performance regardless of the merge, it’s time to focus on your best sources. Look at your best performing sources: Can you expand your use of those sources? Can they find additional universes with similarly performing names? If so, increase the number of names from those sources.
Now take a look at your lower performing names: can you get adequate coverage of those names from some of your higher performing list sources? If so, move the business to them.
This way you’ll be in a better position to optimize your ROI from all your list sources.
Natascha Lee is vice president of marketing for Westminster, CO-based co-op NextAction.