Kanna removes nodes when parsing HTML

I'm using Kanna to traverse a HTML of a website looking for <script type="application:ld+json">.

    func scrapeRecipe(from data: Data) {
        
        let html = String(data: data, encoding: .utf8)
        
        do {
            let document = try HTML(html: html!, encoding: .utf8)
            let script = document.xpath("//script[@type='application/ld+json']//text()")
            
            for recipeJSON in script {
                let data = recipeJSON.text?.data(using: .utf8)
                let json = try? JSONSerialization.jsonObject(with: data!, options: [])
                
                
             //Handling fetched data here   
                
            }
            
        } catch {
            //Handle Error
        }
    }

The issue I am coming across, that on some websites, the let document = try HTML(html: html!, encoding: .utf8) step somehow removes the node notations, but doesn't on others.

The case in point: the original HTML:

<!DOCTYPE html>
<html lang="en-US">
<head>
    <meta charset="UTF-8"/>
    <meta name="viewport" content="width=device-width, initial-scale=1"/>
    <meta name='robots' content='max-image-preview:large, index, follow, max-snippet:-1, max-video-preview:-1'/>

    <script>
    (function(w, d) {
        w.adthrive = w.adthrive || {};
        w.adthrive.cmd = w.adthrive.cmd || [];
        w.adthrive.plugin = 'adthrive-ads-1.0.43';
        w.adthrive.host = 'ads.adthrive.com';

        var s = d.createElement('script');
        s.async = true;
        s.referrerpolicy = 'no-referrer-when-downgrade';
        s.src = 'https://' + w.adthrive.host + '/sites/55cb7e3b4bc841bd0c4ea577/ads.min.js?referrer=' + w.encodeURIComponent(w.location.href);
        var n = d.getElementsByTagName('script')[0];
        n.parentNode.insertBefore(s, n);
    })(window, document);
    </script>

    <!-- This site is optimized with the Yoast SEO Premium plugin v15.9.1 - https://yoast.com/wordpress/plugins/seo/ -->
    <title>Apple Salad with Candied Walnuts and Cranberries | RecipeTin Eats</title>
    <meta name="description" content="This Apple Salad is a celebration of textures and flavours! Crispy juicy apple, leafy salad, candied walnuts, plumped cranberries with a vinaigrette."/>
    <link rel="canonical" href="https://www.recipetineats.com/apple-salad-candied-walnuts-cranberries/"/>
    <meta name="twitter:label1" content="Written by">
    <meta name="twitter:data1" content="Nagi">
    <meta name="twitter:label2" content="Est. reading time">
    <meta name="twitter:data2" content="5 minutes">
    <script type="application/ld+json" class="yoast-schema-graph">{ ... }</script>

And the parsed document.text returns for the same section for the document:


(function(w, d) {
    w.adthrive = w.adthrive || {};
    w.adthrive.cmd = w.adthrive.cmd || [];
    w.adthrive.plugin = 'adthrive-ads-1.0.43';
    w.adthrive.host = 'ads.adthrive.com';

    var s = d.createElement('script');
    s.async = true;
    s.referrerpolicy='no-referrer-when-downgrade';
    s.src = 'https://' + w.adthrive.host + '/sites/55cb7e3b4bc841bd0c4ea577/ads.min.js?referrer=' + w.encodeURIComponent(w.location.href);
    var n = d.getElementsByTagName('script')[0];
    n.parentNode.insertBefore(s, n);
})(window, document);


    
    Apple Salad with Candied Walnuts and Cranberries | RecipeTin Eats
    
    
    
    
    
    
    { ... }

... is what I want to extract, but with the nodes gone I cannot call on this block. I've abbreviated it for clarity of this post.

Has anyone else experienced this with only some websites when using Kanna? The above swift code works fine with 9 out of 10 websites.

Kanna removes nodes when parsing HTML

0 Answers