Scraping google search for sentences based on keywords in google sheets document

user_13954152 0
Viewed 203

I wish to scrape google search results (only the description text) for sentences based on keywords in a google sheets document.

All words will be in column A.

Example of some of the data:
enter image description here

What I require to do is to search google for: "埴生の宿" jp blog and then go down the search results and copy all sentences that are longer than 14 characters and shorter than 30. The sentences must be between。 .or the 、 characters or at the start of the description and can include、 as long as it stays within the character limit.
Once a sentence has been scraped it puts the sentence into the adjacent excel row alongside the keyword. It will repeat this 10 times searching for 10 different sentences for the same word up until column K, then move onto the next word and provide 10 sentences etc.

Is this task possible via google sheets? Or what would be the quickest and easiest way to complete this? There are 40,000 words in column A, so I would need to fetch 400,000 sentences.

web-scraping google-sheets javascript
1 Answers

Your best option is likely to write a simple script in Python/similar language which makes use of a SERP or Search Engine Results Page API such as SerpStack to get the descriptions in an easily further searchable way.

You then use your script to read from the CSV/whatever file, search, find and record sentences.

Only problem with this method is it is slow - I’m sure others here can recommend faster APIs and searching methods.

user_138477050
Related