Puppeteer Sharp - get html after js finished running

user_10139243 0
Viewed 1897

I am using .net core 3.1 and Puppeteer Sharp 2.0.4. I want to get the full page HTML from a web page after the JavaScript has finished running. This is my code:

await new BrowserFetcher().DownloadAsync(BrowserFetcher.DefaultRevision);
Browser browser = await Puppeteer.LaunchAsync(new LaunchOptions
{
    Headless = false
});
var page = await browser.NewPageAsync();
page.DefaultTimeout = 0;
var navigation = new NavigationOptions
{
    Timeout = 0,
    WaitUntil = new[] {
        WaitUntilNavigation.DOMContentLoaded }
};
await page.GoToAsync("https://someurl", navigation);
content = await page.GetContentAsync();

It looks like the content variable does not have the HTML after the JS finished running. Any advice on what I should change to make it work?

puppeteer-sharp puppeteer web-scraping c#
1 Answers

Just replacing navigation with WaitUntilNavigation.Networkidle2 worked to wait until Javascript is finished to excute.

using PuppeteerSharp;

await new BrowserFetcher().DownloadAsync(BrowserFetcher.DefaultRevision);
Browser browser = await Puppeteer.LaunchAsync(new LaunchOptions
{
    Headless = true // false if you need to see the browser
});
var page = await browser.NewPageAsync();
page.DefaultTimeout = 5000; // or you can set this as 0
await page.GoToAsync("https://www.google.com", WaitUntilNavigation.Networkidle2);
var content = await page.GetContentAsync();

Console.WriteLine(content);
user_41835950
Related