2017-06-05

Web crawlers

Web crawlers (web spider) is a search engines software use to traverse the Internet and to save each of the individual pages passed by along the way. Search engines then use additional software to index each of the saved pages, creating a database containing all the words in the pages.

Internet search engines like Google, Yahoo and Bing can search the Internet on topic and return a list of results so quickly. Obviously it would be impossible to search all the Web pages for each request in real time.

Search engine indexed Web pages into highly optimized databases and search matches the keyword each search.

No comments:

Post a Comment