Evolution and revolution of search engines
But soon after, in December 1993, a new robot was born. It was called “spider.” Spiders have achieved a much higher level of precision by indexing all of the text on a web page. Older crawlers only indexed URLs and page titles, which meant some important keywords might not be indexed. This significantly improved the relevance ranking of their results and was therefore the first major step in the development of the major search engines that we all know today.
Shortly after the spider we saw several large cannons appear. In 1994, the world-famous Yahoo was founded at Stanford University. Yahoo was founded by two students, David Filo and Jerry Yang. Basically, in the beginning, Yahoo was just a list of these kids’ favorite websites. However, it has quickly become the most popular website directory thanks to its simple and intuitive interface. Because all websites were reviewed by a human, Yahoo could only index about 1% of the web. During this time, Altavista became the fastest growing spider-based search engine, indexing up to 10 million pages per day.
Until then, there are two different types of search engines: “author-driven” search engines like Altavista and Excite, where results are sorted by keyword relevance, and “publisher-driven” search engines like Yahoo, where users manually submit web pages to your index. . .
At the end of 1997, the most popular and best-known search engine to date was born at Stanford University: GOOGLE Google has a different way of classifying its websites. The Page rank system was used. They simply ranked websites higher in the results based on the number of links pointing to a particular website.
Of course, the content of the page had to be relevant to the keyword entered in the search field. But essentially, Google has developed what could be called a voting system. Therefore, a website with lots of backlinks or votes will rank higher. A backlink is when someone else places a link on your website that points to another external website. Today, Google has over 80% of websites in its index, which is impressive.