How to read text inside a tag

Question

I need to scrape this page: https://www.arabam.com/ilan/galeriden-satilik-lamborghini-gallardo-lp-560-4/mini-motors-dan-2009-gallardo-lp560-4-seramik-lift-bayi-boyasiz/14934711

If you scroll down you'll see this

I scroll down the page and then get xpath for this. This is the xpath: //div[@id="js-hook-description"]//p/text

And this is the code

const results = xpathT.fromPageSource(data).findElements(rest);
    
    //console.log("The href value is:", results[0].getAttribute("href"));
    console.log(`Your full text is "${results[0].getText()}"`);
    if (results.length > 0) {
      let _results = [];
      if (path.includes("href", 0)){
          
          for (let r of results) {
              
            _results.push(r.getAttribute("href"));
          }
      }
      if (path.includes("text", 0)){
          //console.log("inside");
          //console.log(results);
          for (let r of results) {
             console.log(r.getText());
            _results.push(r.getText());
          }

When I simply print results it gives me this:

Your full text is "LAMBORGHİNİ GALLARDO LP560-4
,2009 MODEL - 38.000 KM
,DOĞUŞ OTO BAYİİ ÇIKIŞLI
,
,AİRMATİC (LİFT)
,SERAMİK FREN
,GERİ GÖRÜŞ KAMERASI
,PADDLESHİFT (F1)
,2 BÖLGE KLİMA
,DERİ KOLTUK
,Bİ-ZENON FAR
,YAĞMUR SENSÖRÜ
,CD-USB-AUX-MP3
,
,?
,BOYA - HATA - TRAMER - HASAR KAYDI  YOKTUR
,
,ARACIMIZIN TAMPONLARI DAHİL BOYASIZ
,
,YEDEK ANAHTARI MEVCUTTUR
,?
,DETAYLI BİLGİ İÇİN LÜTFEN ARAYINIZ
,
,

user_12412595 · Answer

You can use page.evaluate to get the innerText property of any DOM elements. In case you need the text by paragraphs you should use the proper CSS selector for the

elements, in this case it is: #js-hook-description > div > p. The matching elements can be collected with the page.$$ method (it is the same as document.querySelectorAll() in the Page's context), then these elements can be iterated over (see a for..of and an Array.map variation below), in each iteration the innerText is retrieved and also a String.trim() is applied to clean the paragraphs from linebreaks (e.g.: ).

// full text content into one string
  const fullText = await page.evaluate(el => el.innerText, await page.$('#js-hook-description'))
  console.log(fullText)

// each paragraph into an array element I.
  const textArray = []
  const paragraphs = await page.$$('#js-hook-description > div > p')
  for (const p of paragraphs) {
     const actualPara = await page.evaluate(el => el.innerText.trim(), p)
     textArray.push(actualPara)
  }
  console.log(JSON.stringify(textArray))

An alternative solution can be done with page.$$eval and Array.map:

// each paragraph into an array element II.
  const alternativeSolution = await page.$$eval('#js-hook-description > div > p', paragraphs => paragraphs.map(p => p.innerText.trim()))
  console.log(JSON.stringify(alternativeSolution))

Full text output:

LAMBORGHİNİ GALLARDO LP560-4 2009 MODEL - 38.000 KM DOĞUŞ OTO BAYİİ ÇIKIŞLI AİRMATİC (LİFT) SERAMİK FREN GERİ GÖRÜŞ KAMERASI PADDLESHİFT (F1) 2 BÖLGE KLİMA DERİ KOLTUK Bİ-ZENON FAR YAĞMUR SENSÖRÜ CD-USB-AUX-MP3 ? BOYA - HATA - TRAMER - HASAR KAYDI  YOKTUR ARACIMIZIN TAMPONLARI DAHİL BOYASIZ YEDEK ANAHTARI MEVCUTTUR ? DETAYLI BİLGİ İÇİN LÜTFEN ARAYINIZ 0533 239 22 77

Array output line-by-line:

["LAMBORGHİNİ GALLARDO LP560-4","2009 MODEL - 38.000 KM","DOĞUŞ OTO BAYİİ ÇIKIŞLI","","AİRMATİC (LİFT)","SERAMİK FREN","GERİ GÖRÜŞ KAMERASI","PADDLESHİFT (F1)","2 BÖLGE KLİMA","DERİ KOLTUK","Bİ-ZENON FAR","YAĞMUR SENSÖRÜ","CD-USB-AUX-MP3","","?","BOYA - HATA - TRAMER - HASAR KAYDI  YOKTUR","","ARACIMIZIN TAMPONLARI DAHİL BOYASIZ","","YEDEK ANAHTARI MEVCUTTUR","?","DETAYLI BİLGİ İÇİN LÜTFEN ARAYINIZ","","0533 239 22 77"]

How to read text inside a tag

1 Answers