Beneath the surface of search

	KPI #2: UNIQUE PAGES

Just how many pages does your Website have? This is a critical metric for establishing a yield management foundation.

You may be tempted to approximate the number of pages using a product count. This will most likely significantly understate your actual pages. Or you might want to use a search engine's reported index. This will return mixed results, as each engine indexes dramatically different numbers of pages for each Website.

But search engines are in the business of discovering pages. So we recommend using the number of pages that the bots (such as Googlebot) are able to crawl over a 30- to 60-day period as the most useful proxy for how many unique pages are available on your site. (This assumes that your URLs are in a condition that permits bots to crawl deeply. If they are not, you may need to resort to an approximation.)

As a comparison, the average merchant in our research had roughly 73,000 uniquely crawled pages.

	KPI #3: PAGES YIELDING TRAFFIC

What percentage of those pages yield search traffic? Is it 10%? 90%? This ratio essentially dictates the length (the X axis) of your unbranded-keyword long tail and suggests remaining potential.

If you do not know your number of unique pages or yielding pages, you can use the matrix at right for an estimate. It was developed using data from the average merchant in our research and illustrates the inverse relationship between page yield and brand/nonbrand traffic. Simply seek your current level of brand and nonbrand traffic to estimate how many pages are accounting for that traffic.

The average merchant in our study had 14% of pages yielding traffic.

	KPI #4: KEYWORDS-PER-PAGE YIELD

Now that you have an idea of your page-yield rate, how many keywords does each of those producing pages yield over the course of a month? Two keywords? Ten?

The keyword-yield KPI is responsible for creating scale. That is, the more keywords each yielding page attracts or targets, the longer your tail.

The average merchant in our study found 2.4 keywords produced per yielding page.

	KPI #5: VISITOR-PER-KEYWORD YIELD

This KPI is basically determined by how highly a page ranks for a keyword; it tells you how much traffic each keyword drives. This metric determines the height or the thickness of your long tail.

The average merchant in our study experienced 1.9 visitors per keyword.

	KPI #6: INDEX-TO-CRAWL RATIOS

The first step in the search-conversion funnel is converting a crawled page to an indexed page.

The average merchant in our study saw a 3:1 index-to-crawl ratio in Google. That is, for every page that was crawled, there were three in the index, as strange as it sounds. If a merchant finds the index shrinking to, say, a 1:2 ration, that may be a signal of crawl pattern changes, or perhaps pages are being shifted into the supplemental index and are therefore much less likely to appear on results pages.

	KPI #7: ENGINE YIELD

Each search engine has a different audience size. How do you fairly compare the referral traffic you get from each? Simply calculate how much referral traffic each sends for every page it crawls or consumes. Compare engine by engine. What we have found is that MSN and Yahoo! tend to crawl a lot more pages, but the yield per crawled page from Google is typically significantly higher.

	Reconstructing the tail

Once you have the metrics for these KPIs, you can create a vivid picture of your natural-search performance and make smarter decisions. For instance, brand and nonbrand traffic composition (KPI #1) provides a baseline on the effectiveness of your channel yield. You can contrast this against your total available pages (KPI #2) and actual yielding pages (KPI #3) to understand the gap between current and potential performance. The rate at which these pages yield keywords (KPI #4) multiplied by the number of visitors driven by each keyword (KPI #5) forms the length and height of your tail. Meanwhile, your index-to-crawl ratios (KPI #6) and engine yield (KPI #7) help quantify your site's effectiveness at converting Web pages into units of “search produce.”

Let's say the number of your pages that yield traffic is low (less than 10%). It may then be time for some basic, scalable URL rewriting to maximize the flow of “link juice” through your internal link text.

On the other hand, if 80% of your pages are yielding traffic, but only at a rate of one keyword per page and one visitor per keyword, it may be time to conduct some auto-generated title-tag tests to scalably lift rankings of thousands of pages by incorporating more relevant keywords per page. Each situation is unique and requires a custom approach to optimization.

The secret to capturing the long tail of natural search is this: Fully maximize the page yield of your Website. That is, develop a metrics-driven optimization process that empowers each page to be viewed as the authority on the subject, whether the subject is “furniture slipcovers” or “men's reindeer sweater.” Empowering thousands of pages to yield natural-search traffic may seem daunting, but it can be achieved by focusing on these new KPIs, managing your NSO effort against these yield metrics, and committing to an iterative NSO testing program.

Increasingly, the natural search game revolves around getting your brand associated with unbranded terms. While these new KPIs do not change the basics of NSO, they can provide the context and management discipline that enable you to make more-informed decisions how to best spend time and budget optimizing your long tail.

---

Brian Klais is vice president, search for Netconcepts, a Madison, WI-based Web marketing agency that offers search optimization services.