The Search engine Ranking Algorithm

Here’s something that drives people crazy about SEO: You can’t ever be 100 percentsure that what you’re doing will be rewarded with the rank and the listing you want.This is because the search engines keep their internal ranking mechanism, even the criteriaby which the ranking is determined, under wraps. Welcome to the secret formulaof SEO: The Search engine Ranking Algorithm.

The algorithm is the formula that a search engine uses to determine its ranks. It’s a way of sifting through a multitude of factors, including keyword repetition and page titles, inbound links, and even the age of the site. Some elements have more weight, meaning that they are considered to be more important in determining rank, and some have less. Each search engine uses its own algorithm to determine which results to show and in which order. And each search engine changes its algorithm from time to time, often without so much as a friendly warning. So, the truth is this: You will never really know exactly how Google works.

Imagine if other forms of marketing worked this way! What if you couldn’t rely on alphabetical order in the Yellow Pages anymore? What if the TV networks chose to air only the bits of your ad that they felt were most important? What if your billboards were periodically relocated without your consent? We’re so glad you’ve got a good head on your shoulders because, now that you’re doing SEO, you will have to find a balance between keeping up with the algorithm and keeping your sanity.

Why do the search engines guard their algorithms so closely? Because, first and foremost, they value the searcher’s experience. If MSN published a guide called Instructions for Ranking #1 on Our Search Engine, of course you’d use it. And so would everyone else. Then all of the results on MSN would become so manipulated by site owners that relevance would disappear—investment sites could rank high for "grape bubble gum" on purpose—and searchers would drop the engine like a big useless hot potato. Even without a manual, the little bits of algorithm that people figure out themselves often get so abused that the search engines eventually devalue them.

How do you find the balance between seeking the Eternally Unknowable Algorithm and making sure your SEO efforts are effective? Matt Cutts, the popular blogger and Google employee who sometimes indulges his SEO-obsessed readers with tantalizing bits of inside information on Google’s algorithm, says, "Most of the right choices in SEO come from asking, What’s the best thing for the user?" Bringing targeted users to your site is, of course, the point of SEO, and that’s the reason we made you clarify your audience and site goals before we started talking about how the search engines work.

Some one asked Danny Sullivan, probably the best known and most respected authority on search today, what he considers to be "Eternal" about SEO. His answer: "Good HTML titles, good body copy, great content, ensuring that your site doesn’t have roadblocks to crawling—these have worked for nearly a decade." Notice he didn’t mention anything about chasing the algorithm.

(In the next post, we will work on, "PageRank"…stay in touch).

Search Results Are Blended

If you’ve spent much time searching, you have probably noticed that the search engines are not displaying one set of homogeneous results. Most search engines take the "chef’s salad" approach, displaying a mix of robot results, directory listings, and pay-per-click (PPC) ads. Your site is probably already represented by most of the types of results we’re about to discuss. Knowing what each type looks like and where they come from is the first step in being able to influence your own listings in a positive way. You learned about robot results earlier; here are the other types of results that are available to searchers.

Directories

Unlike those robot search engines, directory listings are often compiled by humans. Whether these humans are editors who work for the search engines or the site owners themselves who write and submit their own listings, it is often easy to tell the difference between a directory and a robot search result. Take a look at this robot-generated listing from Google. It’s called a snippet—text slurped directly from the web page and spat out into the search results page.

Directories aren’t likely to come out and find you the way robots will; site owners need to submit to them manually. Sometimes you can purchase a listing, sometimes they’re free, and sometimes you pay for the "privilege" of having your submittal reviewed whether your site is included or not. While it’s a little extra work to achieve directory listings, at least you can be relatively certain that your submittal will be reviewed by somebody and your site will have a fair chance of getting in. This is different from the robots, which do not guarantee review or inclusion.

Pay-Per-Click

No matter how blurred the line between unpaid and paid search gets in the search engine results, you, as the SEO expert, will always know the difference. That’s because, while it’s possible to get listed in robot search engines, meta search engines, and even directories without actually doing anything, you (or someone you delegate) will have to actively implement and carefully manage any pay-per-click (PPC) advertising for your own site. And, of course, there’s that little matter of the checkbook too.

Here’s how PPC Works: You open an account with a PPC engine. You decide which search terms you want your site to be seen for, and you write your own listing (or often, several different listings) to correspond with your chosen terms. Every time a searcher clicks on your listing, you pay the PPC engine a fee. You control the amount you want to spend for each click (your bid), and this is a major factor in the placement of your listing.

PPC is the SEO marketing venue over which you have the most control. It offers you a chance to micro-manage your website marketing by being able to target specific messages to specific terms, and even specific geographical locations. It gives you the opportunity to change your message on a whim, and it provides some of the most conclusive tracking around. Therefore, while PPC is by no means a requirement for good SEO, it’s an Eternally Attractive Option to have available to you.

Site Feeds

Site feeds have been around for years in one form or another, but their methodology is still morphing. Available in various forms, they are Eternally Helpful for large or frequently updated sites. Just as you may use a feed to be notified of your favorite blog or news topic, the search engines use site feeds to sit back and receive information from websites without sending spiders out to constantly gather, gather, gather. Feeds work well for regularly edited websites such as blogs and news sites (feeding the content of their daily posts) and online sellers (feeding up-to-the-minute commercial information such as product descriptions and prices). You may also have heard of trusted feed or paid inclusion programs where search engines allow certain "trusted"—and, usually, paying—websites to send the engines regular updates. Generally these types of listings get thrown into the mix with robot-gathered sites and have to fend for themselves, with no special status in the ranking algorithms.

Meta Search Engines

Some people are comparison shoppers, flitting from store to store to review all the merchandise before making a decision. For people who like to compare search results,meta search engines make it easy to review listings from different search engines in one screen—no flitting from site to site necessary. Simply put, meta search engines compile and display results from several search engines and rank them according to their own algorithms. You can’t use SEO to improve your presence on meta search engines directly; if a meta search engine like Mamma.com or Dogpile.com is using Google results, the way to do better on the meta search engine is to do better on Google.

Robots Deliver

We’re going to start with the basics of how the search engines work, and a major component of this is a robot, or spider, which is software that slurps up your site’s text and brings it back to be analyzed by a powerful central "engine." This activity is referred to as crawling or spidering. There are lots of different metaphors for how robots work, but we think ants make the best one. Think of a search engine robot as an explorer ant, leaving the colony with one thought on its mind: Find food. In this case, the "food" is HTML text, preferably lots of it, and to find it, the ant needs to travel along easy, obstacle-free paths: HTML links. Following these paths, the ant (search engine robot), with insect-like single-mindedness, carries the food (text) back to its colony and stores it in its anthill (search engine database). Thousands and thousands of the little guys are exploring and gathering simultaneously all over the Internet. If a path is absent or blocked, the ant gives up and goes somewhere else. If there’s no food, the ant brings nothing back.

So basically, when you think of a search engine, you really need to think of a database that holds pieces of text that have been gathered from millions of sites all over the web.

What sets that engine in motion? A search. When a web surfer enters the term "grape bubble gum" into the search engine, all of the sites that might be relevant for that term are brought to the forefront. The search engine sifts through its database for sites containing terms like "grape growers," "stock market bubble," and "gum disease." It uses a secret formula—a.k.a. search ranking algorithm—to sort the results, and in a fraction of a second, a list of relevant sites, many containing the exact phrase "grape bubble gum," will be returned in the results page.

There are lots of things that factor into the way robot search engines determine the rank for their main search results. But, just for a start, in order to be in the running for ranks, you need to provide HTML text to feed the search engines and HTML links as clear paths to the food. Keeping those robots well-fed and happy is going to be one of the biggest priorities

What Is SEO?

How many of you are reading this blog because you want a #1 rank in Google? Yeah, we thought so. As SEO consultants, we know how good it feels when your website makes it to the top of the heap. Listen, we sincerely hope you get your #1 Google rank, but it won’t help you if it’s bringing in the wrong audience or pointing them to a dead-end website. So don’t think of SEO as just a way to improve your site’s ranking.

The term Search Engine Optimization describes a diverse set of activities that you can perform to increase the amount of targeted traffic that comes to your website from search engines (you may also have heard these activities called Search Engine Marketing or Search Marketing). This includes things you do to your site itself, such as making changes to your text and HTML code. It also includes communicating directly with the search engines, or pursuing other sources of traffic by making requests for listings or links. Tracking, research, and competitive review are also part of the SEO package.

SEO is not advertising, although it may include an advertising component. It is not public relations, although it includes communication tasks similar to PR. As a continually evolving area of online marketing, SEO may sound complicated, but it is very simple in its fundamental goal: gaining targeted visitors.