Live From the Lenser Summit: Is Your Site Index Working?

Sep 25, 2008 8:27 PM  By

San Rafael, CA–You could have an e-commerce site with more than 100,000 items on it. But if your index isn’t working, you could become the great unknown in cyberspace.

How can you tell if Google is indexing your site?

During a session at the Lenser Summit, Lenser partner Bill Nicolai explained that you can simply do a Google search of “site:(url)” and see how many results come up. “The rule of thumb is there should be one page for every product you offer,” Nicolai said.

And if the index is lower than that? Most likely you have a problem with your site architecture, and it’s keeping Google’s spiders–and other search engines–from crawling your site.

So how do you fix that? You’re going to have to bring in a site developer to re-index your site. And even after that’s done, Nicolai says it could take up to 90 days for those index changes to show in search engines like Google’s.

You can do two things to remedy the situation. First, send your site feed to Froogle (which has been known as Google Product Search since April 2007). This could be a panacea for your site, as consumers searching for a product could still find you in the comparison-shopping engine, Nicolai said.

Then, you should make certain symbols and phrases such as #, &, cgi-bin and jsp are not a part of your product URLs. These are spider traps, and keep them from seeing your page and making them a part of the search engine.

And of course the keyword and phrases should be used within the URL, Nicolai noted. In a search for gifts, gifts.com is number-one in natural search. But it also has the top spot for “woman gift” because of the site’s clean indexing.