Merge/purge for small mailers

Jan 01, 2008 10:30 PM  By

You know you have to combine multiple lists and run them through software to single out duplicate records before a catalog mailing. But the merge/purge process is so much more than merging all lists together and purging duplicates. While most of these merge/purge tips and techniques are geared to new or smaller merchants, larger mailers can also make use of them.

Ideally, processing is an impartial tool to output not just data, but information you can use to make better, more profitable mailing decisions. Start with the cost to do the data processing. Always talk to your service provider about the different pricing options based on your particular data needs. If the cost is negligible, input as many records as you can into your merge/purge. This includes all customers, catalog requestors, ship-to, gift recipients, sweepstakes entrants, inquirers, loyalty club members, sister-company names, and customers who have a credit card from your company. (It also assumes compliance with your organization’s privacy policy.)

These lists on your company database or house file can be as specific as house customers and house prospects.

The fundamental reason to run a merge/purge is to identify and omit any duplicate records from outside list rentals. The duplicates can be house file to house file (such as a duplicate found as a catalog requestor and as a recent buyer) or a duplicate between a rental record and another rental record (called a rental multi).

If your data processing pricing is structured so that you pay for all input records, you may want to identify only the universe of records that you eventually want to mail. Doing so limits the quantity input and keeps costs down.

If you’re not sure what’s best for your organization, ask your data service provider to run scenarios and help you identify the options that are most economical. Make sure you review the scenarios for both the slower seasons and the busy seasons.

Running a merge/purge produces many reports that you should become familiar with and use to your advantage. The information gleaned helps to do three things: 1) omit potentially lower performing segments; 2) identify stronger performing segments; and 3) better manage data processing expenses.

Here are some of the reports available from the merge/purge and what you can use to make your mailings more successful — whether saving costs or increasing productivity.

  1. The NCOA reports

    Part of National Change of Address processing includes standardizing all addresses. NCOA mailability scores are parsed 1 through 5. A mailability score of 1 is an accurate address defined by the U.S. Postal Service; 2 is probable delivery; 3 is possible delivery; 4 is most likely undeliverable; and 5 is a drop/insufficient/don’t mail.

    Review these reports and corresponding quantities. You may want to do further research into score 3 — possible delivery. Usually the scores are broken out with additional attributes explaining why the record is not scoring higher. If the explanation references inconsistent address information, such as wrong surname or middle initial, you may want to mail the records with “current resident or” in the first line of the address rather than dropping the records from the mailing.

    Another alternative to omitting the records is to include additional endorsements using the “address service requested” service. This service forwards the recipient and has the USPS return the information to you. There are USPS fees associated with this, but finding best customers and keeping your database current cuts costs in the long run and maintains contacts with buyers.

    If you’re not familiar with the different endorsement services, contact your local USPS representative or speak with you data service provider. If you want general information, look at USPS.com and click on “business” and then “address verification.”

    Many companies fail to use the NCOA reports to their advantage. Don’t be one of them. Make sure you are updating your database with all NCOA information. You are wasting money if you keep paying for the same NCOA changes because you’re not updating your own data.

    How do you know? Look at the NCOA report that shows the matches by recency. The data reveal the month changed and the corresponding quantity. Of course you’ll find quantities from the most recent one to two months.

    If you notice recency of six months, eight months, or more (and you’ve run NCOA for your mailings within eight months) then you have evidence that NCOA changes are not being applied to your own database. The best way to resolve the issue is to have your IT person on a conference call with you and your data processing provider. Let the techie wizards from each company speak their secret lingo and solve the problem. And remember, look at all mailability scores and determine what course of action makes the most sense for your organization.

  1. Mailing “super dupes” (a.k.a. rental hits)

    Most circulation planners say they understand what a super dupe is, but at one time or another, it’s easy to forget how to use the information. For instance, say you’re doing a full merge/purge for a mailing but needing only a small portion of the records for it. When you Nth back a segment (randomly keep a portion of the segment to receive the mailing) or omit older segments entirely, you are dropping the super dupes as well.

    Why does this happen? During the deduping process, if a rental name matches to your house file, the duplicate identified on rental name is dropped and the house record survives. What if the house record is an older catalog requestor or in a recent segment that is not being chosen for this mailing — maybe it’s a current year buyer, but the average dollars spent is not qualifying for the mailing? Now you’re omitting a strong prospective buyer.

    What you need to do is identify the super dupes immediately from the merge. Either have the entire quantity as its own segment, or have the super dupe quantity indicated next to each segment in the mailing. This gives you the choice to mail all the super dupes as a group, or make individual determinations on a segment by segment basis. (Instead of Nth-ing a segment, you decide to mail only the super dupes from that segment.)

  2. Using rental multis

    This is one of the toughest concepts to grasp for both consumer and business-to-business merge/purges. A rental multi is a record that appears on two or more rental lists. Not only is this important to understand, it’s even more important to mail these prospects because they are actively buying from multiple rental lists — and they are not buying from you!

    For instance, let’s say you rent 10 different lists for your Fall Drop 1 mailing and the merge/purge is run on an individual level — meaning the criteria are matched on exact name and address. After the merge/purge processing, one output is identification of all rental multi records.

    Let’s equate a record to a person, John Smith. If John Smith appears on List A, List C, and List J, he is considered a 3x rental multi because his record appears on three different rental lists. You will be mailing the first instance of John Smith with the Drop l mailing, and his keycode will be retained with List C (randomly chosen.) To find out how to mail John Smith the other two times you are allowed, you need to look at the “Rental Multi Matrix” chart to the right, which is one of the standard reports you receive out of the merge/purge.

    Each column of the table is mutually exclusive. The 2x column totals the 1,700 records found on two rental lists. The 3x column totals 450 records that are found on three rental lists, and so on.

    When you’re ready to start the data pocessing for Drop 2, you want to request all 2x+ (the “plus” is important) rental multis. Using the “plus” tells the service bureau to select all records from the 2x through 9x columns. Remember, John Smith is found in the 3x column and you can mail the 3x column three times (which includes the original time with Drop 1.) For Drop 2, this will be the second time mailing the 3x column.

    Fast forward to planning the mailing for Drop 3. Since you cannot use the 2x column anymore because you’ve mailed the 1,700 records twice, you will request the 876 records from the 3x+ rental multis (3x-9x columns.) And remember, Drop 3 will be the third time mailing the 3x column and thus exhausts the usage of the 3x rental multis. For future mailings, continue the sequence for the 4x+ rental multis.

    Are you a rental multi power user? If you are renting hundreds of rental lists, talk to your data service provider about tiered multi reports allowing you to understand the interaction among several groups of rental lists.

    As an example, the rental lists can be grouped into categories such as 1) consistent strong performing rental lists; 2) vendor supplied lists; 3) cooperative databases; 4) average performing rental lists; and 5) test lists. With the tiered multi reports, you may decide you want the 2x rental multis with an interaction with the category 1 lists. Doing so combines the power of a 2x rental multi with the known performance of lists found in category 1.

  3. Merge/purge hierarchy

    Generally, the hierarchy of segmentation follows the RFM (recency-frequency-monetary) cascade. As an example, the 2007, 3x+, greater than $100 average order value segment has a higher priority than the 2007, 3x+, less than $100 AOV segment. But what happens when you need to mail a smaller portion of the file and you don’t want to pay to input a universe of names into the merge/purge? You can reduce costs and minimize on back-end selections with a performance-based merge/purge hierarchy.

    Looking at the segmentation results, sort the segments best to worst, and assign the merge/purge priorities starting with 1 (the highest) to correspond with the best segment; 2 with the next best; and so on. To keep costs down, include only 20% more names into the merge than you will mail. If you historically lose more than 20% in the merge, then adjust input accordingly.

    Once the merge/purge is complete, simply cumulate the segment totals, taking advantage of performance hierarchy, and cut off at the desired quantity. Easy, inexpensive, and strategic!

  4. Hotlines

    Is it more important to give hotlines (recent activity) the highest priority in the merge to retain the surviving duplicate within the hotline (hotline matches to a 2007 buyer, or 2006 or a catalog requester), or the lowest so the surviving record retains in its segmentation? It depends: Some companies put hotlines after the house file segments to reveal true first-time buyers.

In this instance, a hotline that matches any segment above it will survive within the segmentation and be dropped from the hotlines. So any records that do not match the above house file will remain as hotlines and are recognized as first-time buyers. Now you can target this group with bind-in cards, perhaps a different cover version, or other marketing opportunities.

The other option is to put hotlines as the highest priority above all segmentation. This retains all records within the hotline segment and drops matching records elsewhere.

One reason to choose this methodology is based on the segment performance of a hotline buyer. For example, consumer catalogers find recency is the strongest predictor of response. The merge/purge reports give you useful information to productively manage the mail plan.

Depending on your strategies, use hotlines to increase the productivity of your mailing.

Try at least one of these techniques with your next mailing. Use NCOA for more than just updating addresses. Strategically determine the merge/purge hierarchy with hotlines. Mail the super dupes. And use your rental multis to increase performance and save on rental costs.


Gina Valentino is the owner of Hemisphere Marketing, a catalog consultancy based in Kansas City, MO.

Table 1: Fall Drop 1 Rental Multi Matrix
LIST 2X 3X 4X 5X 6X 7X 8X 9X+
List A 90 70 30 5 0 0 0 0
List B 300 200 100 75 50 10 5 1
List C 200 15 10 7 5 3 2 1
List D 120 70 45 25 15 5 1 0
List E 100 10 2 0 0 0 0 0
List F 70 10 3 0 0 0 0 0
List G 325 30 10 5 0 0 0 0
List H 285 20 5 0 0 0 0 0
List I 100 10 5 0 0 0 0 0
List J 110 15 1 0 0 0 0 0
Total 1,700 450 211 117 70 18 8 2