blog




  • Essay / How do search engines work? - 1463

    Since the publication of the World Wide Web in 1991, people have used search engines to obtain their information (Berners-Lee). These sources of information have evolved significantly over the past two decades and continue to become more effective. Although most people with a computer use a search engine, many don't know how it works. To begin with, there are two main types of web searches: bot-based and human-powered. Google is a great example of a crawler or “spider” based engine; it automatically obtains information from sites following a predefined algorithm. These algorithms are massive predefined formulas that automatically sort sites' information and metadata based on user input. Additional coding tells the main system to periodically refresh the sites and information found as the programmer desires. After obtaining data from Hypertext Markup Language (HTML), which consists of the symbols or codes that make up the web page, the algorithm then translates the information into an index structure usable by the search engine. This is similar to how a businessman would organize information in a filing cabinet for later retrieval. Usually, these index structures take keywords from the text, measure the size and type of the web page, and determine whether it is commercial, a news article, or just entertainment. Then, these newly created index structures continue to extract various details from the page, such as color and other seemingly trivial features, to create a searchable product (Sullivan). On the other hand, a search engine such as Yahoo! is human-powered and therefore does not rely on automated algorithms to archive data. Instead, human-powered engines are based entirely on user-submitted data. By doing so, information can be more specific and carefully categorized, which generally results in better searchability. Unfortunately, this type of engine comes with