How can I view the full response that a search engine gets from each page to display and index its results?

Aloe 0
Viewed 20

Search engines will generally show a brief description of one of the most relevant sections of a search result. For example, in the image below:

Google search result

As you can see, the description gets cut off due to space constraints. Is there any way that I could see more of that portion of the text without going to the site? And, ideally, is there a programmatic way to do so, such that I can view more of a particular one of these descriptions after scraping a search engine result page?

I am most familiar with Python, so a solution in this language, with Beautiful Soup, for example, would be best, but if it is better to use other languages like Javascript to do so, I am open to answers using other languages as well.

web-scraping python
1 Answers

This is (at my current state of knowledge, so maybe I am wrong) not a question about programming. This is a question:

Does a search engine have more information available as this one displayed?

DuckDuck does not disrupt the displayed core section with ... . Google does, and is probably only suggesting that there is more at the provided link and doesn't have more of content itself.

Maybe you are only a victim of the deception it does?

Both of the results shown in the image below link to the same URL: (https://moz.com/beginners-guide-to-seo/how-search-engines-operate and were obtained using the keywords "How Search Engines Work: Crawling, Indexing, and Ranking"

enter image description here

enter image description here

One possible conclusion from the pictures above is to suggest usage of another search engine to get the summary text not cut by '...' as for example DuckDuckGo ( https://duckduckgo.com/about )

user95mw0
Related