Search Engine Principles

Do you want to understand SEO? You need to be aware of the design of search engines. The following are the main components:

Spider- a browser-like program that downloads web pages.

-downloads web pages just like a web browser. The difference is that a browser displays the information presented on each page (text, graphics, etc.) while a spider does not have any visual components and works directly with the underlying HTML code of the page. You may already know that there is an option in standard web browsers to view source HTML code. 


Crawler– a program that automatically follows all of the links on each web page.

-discovers all links on each page. Its task is to determine where the spider should go either by classifying the links or according to a predefined list of addresses. The crawler follows these links and tries to find documents not already known to the search engine. 


Indexer- a program that analyzes web pages downloaded by the spider and the crawler.

-parses each page and analyzes the various elements, such as text, headers, structural or stylistic features, special HTML tags, etc. 


Database– storage for downloaded and processed pages.

-the storage area for the data that the search engine downloads and analyzes. Sometimes it is called the index of the search engine. 


Results engine– extracts search results from the database.

-results engine ranks pages. It determines which pages best match a user's query and in what order the pages should be listed. This is done according to the ranking algorithms of the search engine. It follows that page rank is a valuable and interesting property and any seo specialist is most interested in it when trying to improve his site search results. 


Web server– a server that is responsible for interaction between the user and other search engine components.

-The search engine web server usually contains a HTML page with an input field where the user can specify the search query he or she is interested in. The web server is also responsible for displaying search results to the user in the form of an HTML page.

Note: Done with familiarization? Ready for the next step? Please check out my next entry :D