Just because the holiday sales season has arrived doesn’t mean that online merchants should take a vacation from testing their Web sites and marketing to reach peak performance.
That message comes courtesy of Matt Roche, CEO of Web testing services provider Offermatica. Of course, he’s got a vested interest in pushing testing that runs right through the holiday crunch. But he makes the point that merchants stand to gain their own benefits from testing during their busiest sales season.
“Not having the ability to make changes in a Web site over the holidays is a fairly shocking constriction,” he says. “It’s endemic; but just because everyone’s comfortable with it does not mean that they should be.”
Roche’s belief runs counter to a widespread holiday tradition among online retailers: something called “IT lockdown”, the notion that whatever Web site you’ve got on November 1 (or October 1, if you’re an early bird) is the one you’re going to live and do business with through to mid-January. Merchants may think that testing will be too complicated at a busy sales period, or might entail more risk and more potential for lost sales.
And then there’s the IT department, Roche says, which can institute its own operative freeze. “Getting all your e-commerce systems working is complex and unstable,’ he says. “Even the best Web sites are only available 98% of the time. And then when the load goes up four times over Christmas, people believe that whatever benefit they might get from changing the site does not overwhelm the risk of taking the site down. If IT had their way, lockdown would exist forever, because it makes their life easier.”
But testing technology has changed a lot in the last five years, and Web operators no longer have to fear that making changes in a site, testing them and then measuring the results will put any real restriction on their daily traffic or run the risk of a systemwide failure.
The importance of testing has increased year round, Roche says, because customer acquisition costs have risen, thanks to the rise of the search engines as the portal of Web entry for many visitors. “Fewer people are typing in a retailer’s www.com address,” he says. “Most people prefer going to Google and typing in the company name as a way of getting to a site. Others are entering a product name, not even a retail brand. That means you have to pay more often to get people to your site. And the costs of keywords and banners are going up.”
“No longer is it sufficient to say, ‘Well, I’m selling products; I can take orders; I’m dong some online and off-line advertising. I’m basically in balance.’ Now you have to compete.”
When retailers do build a capacity for change into their Web sites, it usually involves either pricing or the key products displayed on their home page. But even some of the biggest retailers don’t support the ability to try multiple changes to see which elements work and which don’t—a key feature of testing. “You have to guess up front what things you’re going to want to change,” Roche says. “That becomes primarily an IT problem again. We’re arguing that this holiday testing should be mainly a marketing issue.”
What Roche and Offermatica favor instead is the ability to set up self-contained “slots” on a Web site that can be changed on the fly and target different groups with meaningful tests of an online retail site’s key elements during the holidays—primarily promotions, key products, landing pages and listings categories.
In terms of a baseline promotion, for example, a retailer may believe that free shipping is going to be crucial to maintaining or increasing sales this season. But free shipping has a burn-out factor associated with it, especially as the end of the delivery season approaches. “It’s a popular promotion, but in a lot of cases it doesn’t have the same impact that it used to,” Roche says. “So why not have 90% of the people coming to your site see that promotion, while the other 10% see either an alternative or no promotion? And at the point where those two lines converge, either switch to the alternative or run no promotion.”
In monitoring free-shipping burnout and in other areas of e-commerce, it’s one thing to know that something like free shipping isn’t going well; it’s another, better thing to know which is a couple of options is best and to be able to put that knowledge into action quickly. “We want retailers to stop guessing and start yelling,” Roche says.
For that reason, retailers should also be testing a variety of landing pages during the holidays to make sure their visitors are clicking through to pages optimized for the highest possible conversion rates. The page’s design and look can also contribute a 20% to 30% uplift in conversions, so pay attention to graphical changes.
Online merchants should also experiment with different systems of product categorization on those pages, to find out if their products are arranged in the way most customers want to shop. For example, a greeting card retailer might want to organize products by season and holiday, or might find that shoppers convert better when given a first view of cards separated into paper and online greetings.
Another area ripe for holiday testing is a retailer’s gift suggestions. Here merchants often trust their gut or their past experience, but changing markets may have rendered both of those unreliable. “First, you want to find out whether gift suggestions affect sales in your particular environment, because they don’t always,” Roche says. “If so, then what you think will be a great gift idea is a good place to start. But the sooner you can move to what people actually want to buy and give, the better.”
That sounds simplistic, but Roche says too many online retailers treat the gift suggestion component of their holiday Web site like pages in a printed catalog. “It achieves this ethereal, otherworldly quality as inviolate,” he says. Instead they should be telling people what to buy, but then as soon as the response starts coming in, they should incorporate that feedback. Testing multiple gift suggestions lets them sharpen their initial guesses about the hot products sooner.
Effective Web testing requires a certain critical mass of data to produce good decisions, but the holiday shopping season is only three and a half weeks long, so the opportunity for running tests is foreshortened. Luckily, the increased traffic most online retailers see during the holidays means that they can test multiple offers, pages, promotions or gift ideas for a fairly short time and still rack up the necessary amount of information.
That’s particularly true if the purpose is to detect a current trend, not to construct a Web site that will stand through the ages. “If you’re splitting a test between two promotions and you see one consistently running ahead of the other, go ahead and make your bet,” Roche says. “The result may have been an anomaly, but it’s more likely the better option.”
Note that all the areas Roche and Offermatica are highlighting are marketing-related. The Christmas/ Hanukah end of the year is not the time to be fussing with your shopping cart. Testing anything that requires a lot of programming work, such as a shopping cart or a registration page, should happen outside the holiday segment. “As far as I’m concerned, Christmas is a marketing season, not an IT season,” he says. “If IT is heavily involved in your marketing, you’re not doing marketing.”
The same goes for offering a slew of new products, including gift certificates for those merchants who haven’t offered them before. The short testing window at the holidays probably makes major new product rollouts an unacceptable risk.
“Stick to presentation, product merchandising, categorization and promotion,” Roche says. “Don’t try to rewrite the holiday shopping whole experience on December 1.”