How to (dynamically) change meta tags before the site is scraped in Angular 2?

user_9314303 0
Viewed 825

I've built a web app using Angular 2, and need to make it more SEO-friendly. I want to dynamically change the values of the head. meta tags in different routes, before the site is scraped by Facebook, Instagram, and Twitter's web scraping robots.

I am able to update the meta tags dynamically as I want, by using the @angular/platform-browser Meta Service:

this.meta.updateTag({ property: 'og:title', content: title }, "property='og:title'");
this.meta.updateTag({ property: 'og:image', content: image }, "property='og:image'");
this.meta.updateTag({ property: 'og:description', content: desc }, "property='og:description'");

So the HTML head is updated correctly both on initialization (from ngOnInit), and on router navigation (from a Subscriber to router.events in app.component.ts)

My issue is that social media's web scraping bots are retrieving the static src/index.html no matter what, before the Angular app adds ANYTHING, including content from router-outlet.

Here is the raw response from Facebook's crawler when scraping my URL:

<!DOCTYPE html>
<html lang="no-NB">
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
  <meta charset="utf-8">
  <title>My Title</title>
  <base href="/">
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <meta name="title" content="My title">
  <meta name="description" content="My description.">
  <meta name="keywords" content="All, Of, My, Keywords">
  <meta property="og:title" content="My Og Title">
  <meta property="og:image" content="https://example.com/ogimage.jpg">
  <meta property="og:description" content="My Og Description.">
  <link rel="icon" type="image/x-icon" href="favicon.ico">
  <script src="https://kit.fontawesome.com/1e146e9a97.js" crossorigin="anonymous"></script>
  <link href="https://fonts.googleapis.com/css?family=Roboto:300,400,500&amp;display=swap" rel="stylesheet">
  <link href="https://fonts.googleapis.com/icon?family=Material+Icons" rel="stylesheet">
</head>
<body>
  <app-root></app-root>
<script src="runtime.da6e4981b1413a95ffba.js" defer></script><script src="polyfills.6282a28eb8950eb48467.js" defer></script><script src="main.74eff1edffffe134e66e.js" defer></script>
</body>
</html>

As you can see, it retrieves a response from my site before anything is loaded - AKA the original, static index.html.

Is there a way around this, or do I actually have to run multiple apps with their own index.html in the same project?

angular seo web-crawler
1 Answers

I've built a web app using Angular 2, and need to make it more SEO-friendly. I want to dynamically change the values of the head. meta tags in different routes, before the site is scraped by Facebook, Instagram, and Twitter's web scraping robots.

This is actually a common limitation of SPA's (Single Page Applications), as the HTML meta-tags are generated on the client. similar question here

Basically, the crawler from Google/FB can't see the tags as it doesn't actually render the page, it only requests the raw HTML and sees the basic index.html that Angular starts with. You can simulate what the crawler sees by using curl to request your page: curl http://myangular.site/about, which will output the HTML that the crawler will index.

There's a few solutions to this:

  • Server Side Rendering (SSR)
  • Don't use a static server
    • When /about is requested, serve the index.html file and add the meta tags onto it.
user_24195840
Related