(Magilla Marketing) JupiterResearch analyst David Daniels is spearheading an industry-wide effort to standardize e-mail metrics, starting with defining terms around delivery. And so far, just about everybody who’s anybody vendor- and trade-group-wise seems to have signed on.
Dubbed the E-mail Measurement Accuracy Coalition, the initiative comes on the heels of various reports revealing the dismal state of e-mail metrics. According to recent surveys, many marketers are measuring little that is meaningful in their e-mail programs, and those who are define fundamental terms vastly differently.
For example, the delivery rate–probably the most fundamental metric in the business–has no standard definition, according to a survey conducted earlier this year by the Email Experience Council, a for-profit organization aimed at promoting standardization in the commercial e-mail industry.
Seventy-nine percent of e-mail service providers surveyed defined “delivered” by deducting all failures from total mailed, while 21% calculated it by deducting only hard bounces–where the address no longer existed–from the total mailed.
E-mail marketers were in even further discord on the “delivered” metric, as 63% defined it as total failures subtracted from total mailed, 11% defined it as simply total mailed, and 10% defined delivered as only those e-mails that made it into the recipients’inboxes vs. their spam folders.
As a result, standardizing deliverability metrics is the E-mail Measurement and Accuracy Coalition’s first task, according to Daniels. He adds that rectifying some of the channel’s inherent sloppiness will be critical.
“A lot of the information that comes back from the ISPs is not accurate, and they do that purposely to throw off spammers,” he says. “So part of it is getting a… measure for how you deal with that noise, and then more importantly, what do you do with all the hard bounces and the failures? Do you keep them in the arithmetic, or do you remove them?”
Currently it’s difficult for marketers to judge the performance of various e-mail vendors because the service providers present their metrics differently. “Clients have come to us and said, ‘We have some of our mail with this e-mail vendor and some of our mail with this [other] e-mail vendor, and we can’t even internally do benchmarks because the two vendors use different measurement methodology,” says Daniels.
He says he understands that because some e-mail service providers will have to change the way they measure their services, the standardization process won’t necessarily go smoothly.
“It’s going to be a challenge to build consensus,” Daniels says. “There are a lot of competing interests, and there are some real costs involved.”
But Jordan Ayan, chief executive of e-mail service provider SubscriberMail, says that any changes involved in reaching an industry consensus should be painful only to vendors that have been fudging their numbers.
“If you’ve been very clean in the way you’ve been reporting data, doing whatever comes out of this will be relatively easy,” Ayan says. “But if you’ve been doing tricks behind the scenes, you’re going to have to come out from behind the curtain. This is long overdue.”
Ayan adds: “There are a lot of people who have very good delivery rates in this industry now. But there are also people who have bad delivery rates who have been reporting good delivery rates.”
Participation in the coalition is open to anyone and free, says Daniels. He expects the group to finish its work and disband by the end of summer. “This is not going to be yet another ongoing coalition,” he says.