When measuring the effectiveness of a promotion, it’s not enough to know the overall response and revenue. What you really need to measure is the incremental order activity generated by the promotion.
Confused? Let’s say that you mail a print catalog, then follow it two and a half weeks later with a remail of the same catalog with a cover change. How many of the orders received subsequent to the second catalog’s in-home date were the result of the second catalog? How many would have occurred even if the second catalog had never been mailed? And for those catalogers with an e-commerce site, how many orders would have come in if neither the first nor the second catalog had been sent? By answering those questions, you are measuring incremental order activity.
Catalogers implicitly acknowledge the existence of baseline order activity when they say that a remail “cuts off the tail” of an initial catalog’s “promotional curve.” Just about everyone, however, is struggling with how to quantify the exact amount of cannibalization that remails have on the initial drops and the baseline response and with how to determine the amount of order activity that Websites are responsible for.
It stands to reason that if you don’t know the level of baseline order activity, you cannot determine the amount of incremental purchases generated by a second mailing — or by an e-mail campaign, merchandise-specific direct mail pieces, field sales force activity, store promotions, or advertisements. Nor, for that matter, can you measure the incremental response of a catalog mailing above and beyond self-generated (or affiliate-driven or search-engine-driven) e-commerce activity. In short, without understanding your baseline response, it is impossible to measure the effectiveness of a promotion.
The retail analogy
Savvy retailers have known for years about baseline purchase activity as a result of free-standing inserts (FSIs) — the inserts that fall out of everyone’s newspapers on Sunday mornings; run-of-press (ROP) ads in the main body of newspapers and magazines; TV and radio spots; and the powerful intrinsic advertising that takes place every time someone walks or drives by a retail location.
These retailers understand that just a fraction of the store traffic that occurs within the response window of a direct promotion is the result of that promotion. They recognize that there is not a guaranteed stimulus/response relationship between a direct promotion and a subsequent purchase. They have learned that direct promotions are diluted and sometimes even overwhelmed by the vast ocean of ongoing mass promotional stimuli.
This lack of a guaranteed stimulus/response relationship between direct promotions and subsequent purchase activity is true even when a response device is included. It is a recognized phenomenon that a significant chunk of customers who take advantage of response devices would have made a purchase anyway.
Today’s catalogers find themselves in the same sort of “open loop” environments as retailers. The advent of multichannel marketing, and especially the rise of the Web, makes it easy for customers — and to a lesser extent, prospects — to generate order activity on their own, regardless of how many promotions they receive.
Beyond the match-back
When it comes to promotional measurement, many multichannel merchants have not kept up with the times. They continue to use the old-school tool of match-back processing. Match-back processing is driven by business rules to allocate order information and generate campaign reports. I would argue that this approach does nothing to address the issue of measuring incremental performance in today’s multichannel world of overlapping promotions.
One cataloger recently learned this first hand. It had tested radically new creative, and the initial results were extremely encouraging. Just to make sure, though, the cataloger decided to track customer behavior beyond the normal response window. To the cataloger’s astonishment and dismay, it discovered that nine months after the test window just about all of the new treatment’s advantage over the tried-and-true promotional package had dissipated! What had happened is that the apparently successful test had shifted demand forward — that is, pulled order activity into the normal response window that would have occurred anyway in the more distant future.
Nevertheless, the cataloger had inadvertently started down the path toward accurately measuring incremental performance in today’s multichannel world of overlapping promotions. It had stumbled upon the fact that a “longitudinal analytical framework” is required to support the continuous creation of knowledge.
In plain English, the first, “base” component of such framework is a robust repository of complete customer history, including:
-
unabridged, atomic-level order and item “demand” transactions and, when appropriate, post-demand transactions such as returns, exchanges, and allowances.
-
all promotional contacts, including mail, e-mail, and phone.
-
scrupulously deduped individual-level data properly linked to the household level (for consumer marketers) or to the site level, with the site-level data subsequently linked to the company level (for business-to-business marketers).
-
the ability to easily re-create past-point-in-time customer views, model scores/segment definitions, business rules, and “time-0 snapshots” for predictive modeling and cohort analysis.
This repository can be defined as a “marketing database.” But it is much more than what most catalogers refer to as their marketing database.
One of the leading operational systems, for example, advertises add-on marketing database capabilities. Among other deficiencies, though, this database offering fails to scrupulously dedupe customers and inquiries, does not support individual-to-household and individual-to-site-to-company-level linkages, and does not maintain complete promotion history. As for other service company solutions, some routinely roll off order, item, and promotional data after specified time periods such as twenty-four months. And most product and service solutions do not include point-in-time capabilities.
The second component of a longitudinal analytical framework is the implementation and subsequent analysis of continuous waves of across-time test treatments, in order to clarify the complex interactions of channels and promotions. All ongoing analysis takes place within the robust repository of complete customer history.
With such an approach, the devil is in the details, and the analytical challenges, as you can imagine, are significant indeed. The payoff, however, is the insight gained into the behavior of customers and prospects, and into the accurate measurement of incremental response to promotional activity.
Jim Wheaton is a principal at Wheaton Group, a database systems and services provider based in Chapel Hill, NC.