Search engines are big business. As the Internet expands daily with more pages and more information (and more junk), a high ranking in search engine results becomes increasingly important to organisations who rely on web traffic to drive their business. A whole industry has grown up around optimising and marketing websites to achieve the holy grail of a top-10 placing on top search engines. Companies hand over thousands of pounds to search engine optimisation services.
When most people think of search engines, they think of one word - GOOGLE. The search engine giant, begun in 1998 by two American students, Larry Page and Sergey Brin (read more about the history of Google) is now worth $50 billion, and its two founders (still only in their mid-thirties) over $1 billion each. Google is now diversifying into software with technology such as Desktop Search and the fantastic Google Earth (articles on the latter are planned for the blog in the near future). Microsoft, it is reported, are concerned that Google is moving into their turf . . . .
Google’s search engine success leads to problems though. Because it is the dominant search engine, it is also the search engine that web designers target when planning their internet marketing strategy. While Google wants to provide the relevant search results that their visitors require, web designers want Google to return search results that include their website. All manner of techniques and tricks are employed by web-designers in an attempt to improve their Google rankings for searches.
Google (and all other search engines) use "robots" or "spiders" (computer programmes that scour the web indexing websites) to examine sites and work out what they are about and whereabouts in the rankings they should appear. Because these programmes are not human-beings, they can be fooled - they don’t "know" what a good website looks like or whether the information on it is valuable or accurate. Google uses the rationale that if other sites are linking to a site then it is probably more useful then a site which isn’t linked to by other sites. This is open to abuse too, however, as people set up "link farms", websites that are nothing more than a series of links to other sites. Google is constantly updating the way that it indexes sites in an attempt to protect the quality of their search results.
Recently, I have begun searching the web using a search service other than Google . . . well sort of. A recent survey of search engines discovered an interesting fact about the leading search engines. If we examine the most popular search engines (in order of usage: Google, Yahoo, MSN and Ask Jeeves), then, on average, only 3% of websites for popular searches (e.g. "cinema") will appear in the first page of results for all of them. So if you search on Google, for example, you may be missing sites that Yahoo considers to be relevant to your search. Because the different search engines use different algorithms for their robots, they rank sites in different ways.
What if you could search all of these search engines at once? That is the idea of metasearch. Metasearch engines search multiple engines all at the same time and combine the results. The most popular metasearch engine in the UK recently has been Dogpile.co.uk. The engine has recently been renamed to WebFetch.com (presumably because of the slightly "messy" implications of the original name).
Searching on WebFetch.com
Let’s perform a search on WebFetch.com for "city learning centre" to see what we get:
When we hit the "Go Fetch" button, we get the following results:
The search results are presented as you would expect on Google. Beneath each search entry, WebFetch also tells you which of the search engines that site was found on. There, sitting at no.1 (!) is our own Wolverhampton City Learning Centre website at www.wolverhamptonclc.co.uk.
If I’m not happy with the combined results, however, I can search each of the engines individually, or even compare the results from each. Above the results are buttons for each of the major search engines. Pressing on one of these brings up a new set of results - just those from that search engine.
Here are the results from Google:
Wolverhampton City Learning Centre appears at no. 5 in that list. Here are the results from MSN:
The theory is that by combining search results from lots of search engines, the results are less likely to be affected by the tricks employed by web designers to achieve a high ranking in one engine (usually Google).
Give WebFetch a try, and see if it improves your search results!
