How do i check for multiple keywords using cheerio?

user_12084859 0
Viewed 34

Currently, I am trying to build a scrapper that searches all <a> tags for specific keywords like "climate" or "environment." Using cheerio, is it possible to look for multiple keywords so that I get results of multiple keywords?

Here is my code-

const PORT = 8000;
const express = require('express');
const axios = require('axios');
const cheerio = require('cheerio');
const { response } = require('express');

const app = express();

const articles = [];

app.get('/',(req,res)=>{
    res.json('Hello World')
})

app.get('/news',(req,res)=>{
    axios.get('https://www.tbsnews.net/bangladesh/environment/climate-change')
        .then((response)=>{
            const html = response.data;
            const $ = cheerio.load(html);

            $('a:contains("climate")',html).each(function(){
                const title = $(this).text()
                const url = $(this).attr('href')
                articles.push({
                    title,
                    url
                })
            })
            res.json(articles)
        }).catch((err)=>console.log(err));
})

app.listen(PORT,()=>{console.log(`server running on Port ${PORT}`)});
cheerio express web-scraping jquery javascript
1 Answers

From the documentation of cheerio, you can use multiple contains selectors similarly you would use them in jQuery.

Logical OR

If you need to match any of the words, just separate the contains selectors with a comma.

$('a:contains("climate"), a:contains("environment")', html)

Logical AND

If you need to match exactly the two words, add the second contains selector right after the first.

$('a:contains("climate"):contains("environment")', html)
user_13719950
Related