-

What are Web Search Engines and how do they Work?

Web search engines as the name suggests are websites that help you to search the internet. Web search engines contain a huge database of websites which they collect by using spiders or robots and make use of this database to display results when someone requests for a search.

What are search spiders/robots?

Search engine spiders/robots are automated programs that are programmed to search the internet. These programs are used by web search engines to retrieve web pages to include in their database.

Generally all internet search engines make use of robots that crawl and retrieve web pages from the internet via the World Wide Web and then send them to the search engine's indexing software. The indexed web pages are then stored in a database from which the search engine extracts results to be showed on the search engine results pages.

Types of Search Engines

1.) Crawler based search engines

Crawler based search engines are those that crawl the web to create lists automatically without any human influence. These internet search engines make use of specialized programs called crawlers or robots that crawl the web and cache web pages. The web pages are then stored in a database after which they are indexed based on a search algorithm. Each search engine may have its own specific search algorithm to index pages. The basic things that search bots take into consideration to rank and list pages include title tags, Meta tags, anchor texts, domain name, file names, directory name and site content among others.

Some popular crawler based search engines are Google, Yahoo!, MSN and Ask Jeeves. These are considered the best search engines over the internet today.

2.) Human edited directories

Directories are search engines that do not make use of search bots and manually edit websites. Directories accept website submissions and then manually go through these submissions and get them listed if found relevant. Some popular human compiled directories are dmoz and Yahoo!

3.) Meta Search engines

Meta Search engines are those which give search listings collected from various other search engines. They do not have their own database or spiders and get results by querying the databases of many different search engines. The benefit of using a Meta search engine is that you can get results from different search engines instead of manually searching those engines. Some popular meta search engines are Dogpile, MetaCrawler and Mamma.

4.) Pay per inclusion search engines

Pay per inclusion search engines are those that do not offer organic listings but list only those websites that offer to join their search program by paying a listing fee. In addition to the listing fee most pay per inclusion search engines also charge an additional fee for every click received. Anyway pay per inclusion search engines do not ensure top ten positions to any website and the price charged is only for inclusion in the search database. Though websites are crawled more frequently which ensures that visitors find the most updated content; more frequent crawling also ensures better search engine optimization.

Currently the number of search engines that offer pay per inclusion services has reduced considerably with only a handful of search engines left. Some most popular search engines that offer pay per inclusion are Entireweb, Lycos and LookSmart.