Flutter WebView: Retrieve ALL fully generated html from a rendered URL(Similar to "Inspect Element")

Viewed 946

I am trying to get the titles of the events from this webpage and put them into a list:

enter image description here

Since using HTTP did not get the fully generated html, I decided to try using flutter_webview_plugin to:

  1. Load the desired URL on a hidden webview
  2. Extract the fully generated HTML
  3. Parse the HTML to get the titles I need

I am using this code to load the webview:

flutterWebviewPlugin.launch(
    "https://www.volunteermatch.org/search/?l=Long%20Grove,%20IL%2060047,%20USA",
    hidden: true);
flutterWebviewPlugin.onStateChanged.listen((viewState) async {
  if (viewState.type == WebViewState.finishLoad) {
    flutterWebviewPlugin
        .evalJavascript("document.documentElement.outerHTML")
        .then((html) {
      print(html);
      //some code to parse html
      flutterWebviewPlugin.close();
    });
  }
});

The problem is that even using JavaScript code to retrieve the HTML after the webview has finished loading does not give me the same HTML that going to "Inspect Element" on Safari would give, and none of the titles are in it either. What am I missing?

1 Answers

Well, now I feel silly.

Since I was running the app on my iPhone, I was using Xcode to check the output of print(html). For some reason, Xcode doesn't print out all of the html string:enter image description here

Xcode just leaves a <...>(bottom right corner circled in red) and decides not to print the rest.

Related