How do Google and Bing index a blazor site

user_6539986 0
Viewed 737

How do Google, Bing and other sites index a blazor site?

When I view the page source of a blazor app, either Server or WebAssembly (WASM), then all I see is an HTML wrapper which loads the app. To see the generated source code I need to use the DOM inspector. This article discusses putting a robots.txt in place (Where is the best place to put Robots.txt in a Blazor Web Assembly project?) but when I create my site index, every detail page it will point to will be dynamically generated by a .razor file.

Do I need to do anything special to ensure that Google, Bing and others can index my site as I want them to?

blazor-webassembly blazor web-crawler
2 Answers

This is not strictly a Blazor issue. Search engine crawlers like Googlebot (Google's crawler) and Bing do not execute WASM code, so they can't index a statically generated site like Blazor does. Although, looking at Blazor's release notes, it seems you can enable pre-rendering and host Blazor on a server, thus allowing HTML content to be rendered on the server and sent back to the user. This type of server side rendering (SSR) allows search engine indexing and increases SEO. Take a look at an example here: https://andrewlock.net/enabling-prerendering-for-blazor-webassembly-apps/

user_40377150

MrMikimn's answer above solves both ends of the problem.

Configuring Blazor to use pre-rendering does two things: Firstly it allows me to make the html source code visible to a stock standard HttpClient GET call, which allows indexers to call and index the page, but also it now allows me to use an API controller to produce the siteindex.xml file which I need to use to guide bots to the right place.

This is done simply by adding an API Controller and setting it to respond to calls for the Robots.txt and sitemap.xml:

[Route("/robots.txt")] 
public string robots()...

[Route("/siteindex.xml")] 
public string sitemap()...
user_65399860
Related