A Professional SEO's Cheat Sheet for Building a Search Engine Friendly Website

A Professional SEO's Cheat Sheet for Building a Search Engine Friendly Website
—by Esoos Bobnar and Stephen Mahaney

The art of creating a highly optimized and search engine friendly site revolves delicately around three important factors:

1. Creating content that search engines can easily crawl and index.
   
   
2. Linking your site together so that PageRank flows correctly and is channeled to your most important pages.
   
   
3. Placing your keywords in the proper locations on your page for optimal ranking effect.

Some sites get one or two of these correct, but it's rare to see the site which has mastered all three. And that's good news for you, because if you fully understand the art of creating a search engine friendly site then you've got a huge advantage over your competition. Get all these factors right and you'll be able to mow 'em down before they even see you coming.

So toss out your bug spray and let's hope you're not squeamish...we're going to get friendly with some spiders—search engine spiders, that is!

Crawlability: Get over your arachnophobia—spiders are your friends

Crawlability is just a fancy way of describing how easy it is for search engine spiders to download your pages, easily access your content, and find the rest of your pages based on how you've linked your site together.

It's very common for sites to put up roadblocks that stop these spiders in their tracks, shutting them out of important areas of a site. What's more, Google now has a multi-tiered crawling strategy which reserves only the most worthy pages for the main index, banishing everything else to the supplemental index dungeon, ignored and forgotten. Put up a spider speed bump and you might see your page drop like a rock out of the rankings.

You want spiders to eagerly gobble up your pages and return to dine often, vigorously crawling all the corners of your site to make sure your pages get indexed and, just as important, stay indexed and well-ranked in the engines.

Here's a checklist of the most important ways to make sure your site is crawlable. Let's keep those spiders fat and happy :)

The Do's...

• Use HTML for content. If you want search engines to index your content, place it in simple HTML text format. Placing your content in Flash, AJAX, JavaScript, frames, or other technologies will make it harder for search engines to index your content.

• Use HTML for links. If you want search engines to follow your links, place them in simple HTML format. Just as with your content, don't hide those links in Flash or JavaScript.

• Make the most of sitemaps. Employ both a regular HTML sitemap and an XML sitemap.

• Domo arigato, Mr. Roboto. Use your robots.txt file to prevent search engines from wasting time indexing unnecessary files.

• Fix your broken links. Use Xenu Link Sleuth and Google Webmaster Central to make sure your site doesn't have any broken internal links that search engines won't be able to crawl.

The Don'ts...

• Don't require form submission. Spiders don't fill out forms or click submit buttons, so don't require them in order to reach pages you want indexed. If you have content behind forms that needs to be indexed, make sure you provide an alternative link for the spiders to crawl.

• Don't require registration, cookies, or search. Spiders can't fill out registration forms. Same with cookies—search engine spiders don't accept them. And don't make spiders use your site search engine to find your site's content either.

  If you must require registration or cookies to view content, the best way to ensure your content gets indexed is to use IP delivery to recognize search engine spiders and let them in the door. As Google says:

  ...Configure your webservers to not serve the registration page to our crawlers (when the User-Agent is "Googlebot"). You can verify that the request is actually from our robot by making sure the IP address is within the range of 66.249.64.0/20. It’s equally important that your robots.txt file permits Googlebot to access your site.
  
  

• Don't serve up meaningless URLs. Be careful with session IDs, calendar links, or other technologies that generate large numbers of useless pages. You risk making it possible for spiders to get a different URL for the same page every time they access your site (such as with session IDs) or to encounter an endless number of links (such as a calendar program which links to a page for every day off into the infinite future).

  If this happens, then spiders are going to spend too much time indexing useless pages on your site and not enough time indexing your important pages. It can also depress the overall PageRank equity of your site by spreading it out over vast numbers of pointless pages.

• Avoid long and complicated dynamic URLs. You know, the ones with lots of ?'s, &'s and ='s. From both a search engine and a user perspective, short and descriptive URLs are ideal. A user should be able to tell what your page is about by reading the URL the search engine has displayed for your page's search engine listing. And, in general, subdirectories are better than subdomains.

• Frames are bad. Don't use 'em.

• Avoid having excessive duplicate content on each page. For a shoe site, you might have a page selling Women's Athens Crocs. You don't need a separate page for every color of Women's Athens Crocs you sell, particularly if each page is identical in content except for the color of the shoe. One page is fine.

Now that we've covered how to get and keep your content indexed by the search spiders, let's look at the way those spiders are directed around your site. Based on its internal linking, every site sends a message to search engines about what pages are the most important.

The question is: are you sending search engines the right message? In other words, are you telling them your About Us page is just as important as your highest converting money page? Let's certainly hope not!

Site architecture: Put the focus where it needs to be.

• Match your keywords to your site. This means using both free and paid keyword tools to find the right keywords to target, as well keyword brainstorming and feedback from your customers (particularly using your site search engine and your server logs).

• Then organize those keywords into related groups. If you had a pet supply site you might organize your keywords into a group of dog-related keywords, a group of cat-related keywords, a group of bird-related keywords, and so on.

  Each keyword group then becomes its own category on your site:

  http://petsupplyhub.com/dogs/
  http://petsupplyhub.com/cats/
  http://petsupplyhub.com/birds/
  ...

• Use your homepage to link to the main page within each category. Within those categories, you then develop your individual pages designed to rank for the various keywords in that group. Ideally, pages within each category will link primarily just to other pages within their category—and tend not to link to pages in other categories.

  This keeps each category tightly themed (i.e. focused primarily on its keyword group) and makes it much easier for search engines to identify which keywords your pages should rank for.

• Reinforce the important landing pages. Make sure every page in each category links back to its main category page and to the overall site's homepage. Reinforce the rankings of your most important product page by linking to them from within each group as well.

• Be consistent in how you link your pages together. Always use the same URL format when linking to your own pages.

• Create a flat site architecture. How you structure your directory tree is not especially important—how you structure your internal linking is.

Clearly, you should know what pages you want to get ranked (product pages, article content) and which pages you don't (About Us page, Privacy Policy, etc.) and focus your efforts accordingly.

Besides targeting your internal linking to promote your most important pages (and reinforce your site's themes) you also want to place your targeted keywords on those pages to increase their chances of ranking well. That's where on-page SEO comes in.

On-Page SEO - Putting the Right Keywords in the Right Places

• Each page should be targeted to a specific keyword phrase. Using modifiers, synonyms, related keywords and long-tail secondary keywords is fine too (in fact it's encouraged), but you should always have a primary keyword or keyword phrase in mind for each page you're optimizing.

• Avoid optimizing more than one page for a given keyword. This just confuses search engines as to which page on your site they should rank for that keyword.

• Give each page a unique title tag, and make sure that your keyword appears in the title tag of that page. The ideal title tag is both keyword-rich and enticing to your customers so they'll click your listing. Title tags are your most important on-page element, so be sure to get them right.

• Keep your titles below 65 characters in length. This is the maximum to use if you want to make sure the complete title gets displayed in the search results on all the major engines. While it's true that character length is not a ranking factor, it's also true that a good looking title in the search results is more likely to get clicked on.

  So, even though some search engines will display more than 65 characters, if you want your complete titles displayed in all the engines, go with the under 65 character limit.

• Give each page a unique meta description tag. Since meta descriptions are not a critical ranking factor, keyword placement is not as important. However, a well-written and customer-oriented description meta tag can greatly increase the number of clicks you get on your listing in the search results.

  Also, search engines will bold the keywords in your description tag that match those used in the searcher's query, automatically drawing the searcher's eye to them and increasing clicks. So be sure to include your keywords in your description (even if they don't impact rankings). And stick to 160 characters or less to avoid having your descriptions chopped off.

• Don't waste your time with keywords meta tags. Simply put: they do nothing.

• Use NOODP and NOYDIR. If you're listed in DMOZ or the Yahoo Directory, using the NOODP and NOYDIR meta tags will keep Google and Yahoo from messing up your carefully optimized titles and descriptions and replacing them with your un-optimized directory listing titles and descriptions. When search engines do this, it can have a significantly negative impact on your click-through rate.

  To prevent Google from using your DMOZ title and description, use:

  
  
  

  To prevent Yahoo from using your Yahoo Directory title and description, use:

  
  
  

  To prevent Microsoft from using your DMOZ title and description, use:

  
  
  

  Or you can just target all of them at once like this:

  
  
  

• All other meta tags are meaningless when it comes to search engine rankings. The only other meta tags you'll ever want to use are the NOINDEX, NOARCHIVE, or NOFOLLOW robots meta tags, and that's only when you want to prevent search engines from indexing, storing, or following links on one of your pages.

• Each page should have a single h1 tag containing the primary keyword phrase you're targeting for that page. This is the main heading for the page and is an important ranking factor. Generally you can just re-use whatever optimized text you're using for the title tag of the page. This means users will see prominently displayed text on your page which matches the title of the listing they just clicked on in the search results. This provides a sense of continuity and a good user experience.

• If appropriate, use h2 and h3 tags for subheadings. Don't reuse the h1 tag—limit the h1 tag to one per page.

• Use your keywords in lists, bolding, and italics where appropriate. But don't get carried away, as these are mostly minor ranking factors. Instead, focus on formatting your page so that it's easy to read.

• Use your keywords in the alt text of your images, especially if those images are also links. Alt text acts as the anchor text for image links, so it's critical that you employ it. However, make sure the alt text is also descriptive and readable, not arbitrarily keyword-stuffed. Text based links are always going to be best, but if you must use image links then make sure they have alt text.

• Optimize your anchor text. Make sure the anchor text (or alt text for image links) you use to link your pages together is both descriptive and contains the keywords you want your page to rank for.

• Don't just rely on your site's navigation menus or sitemap to link your pages together. Make sure your internal pages have links within their main content body linking to other internal pages on your site. But keep those links primarily pointing at pages within that page's own category to keep from diluting the theme of each category.

• Surprise, you're done! Once you've completed all of the above, your on-page keyword placement is finished. Don't worry about tweaking keyword density—just write to appeal to your human readers. From here on out, most of your returns will come from building links to the page and improving the page's sales conversion ratio.

When search engines shifted toward using link based ranking algorithms, the art form of designing effective on-page optimization became slowly submerged by the clarion call to get links. But the critical need for ensuring that spiders find your keywords on the right pages, and in the right locations, has not diminished. If anything, it's grown in importance.

As the volume of pages on the Internet continues to balloon, search engines must continue to become more and more discriminating in choosing which pages to include in their indexes. Therefore, it has never been more important to make it easy for those busy little spiders to do their job not only finding your pages but also categorizing them correctly. Get it right and the rewards can be enormous.

Let's just call it, Arachnooptimizing!