Getting a URL Rewrite error 500 when prerendering

r3plica 0
Viewed 9

Maybe someone can help me out. I have two rules in my web.config, one for pre-render and one for Vue. I have stacked them like this:

<rules>
    <rule name="prerender.io" stopProcessing="true">
        <match url="(\.js|\.json|\.css|\.xml|\.less|\.png|\.jpg|\.jpeg|\.gif|\.pdf|\.doc|\.txt|\.ico|\.rss|\.zip|\.mp3|\.rar|\.exe|\.wmv|\.doc|\.avi|\.ppt|\.mpg|\.mpeg|\.tif|\.wav|\.mov|\.psd|\.ai|\.xls|\.mp4|\.m4a|\.swf|\.dat|\.dmg|\.iso|\.flv|\.m4v|\.torrent|\.ttf|\.woff|\.svg)" negate="true" />
        <conditions logicalGrouping="MatchAny">
            <add input="{HTTP_USER_AGENT}" pattern="googlebot|bingbot|yandex|baiduspider|facebookexternalhit|twitterbot|rogerbot|linkedinbot|embedly|quora\ link\ preview|showyoubot|outbrain|pinterest\/0\.|pinterestbot|slackbot|vkShare|W3C_Validator|whatsapp" />
            <add input="{QUERY_STRING}" pattern="_escaped_fragment_" />
        </conditions>
        <serverVariables>
            <set name="HTTP_X_PRERENDER_TOKEN" value="<removed>" />
        </serverVariables>
        <action type="Rewrite" url="https://service.prerender.io/https://{HTTP_HOST}{REQUEST_URI}" appendQueryString="false" logRewrittenUrl="true" />
    </rule>

    <rule name="Vue" stopProcessing="true">
        <match url=".*" />
        <conditions logicalGrouping="MatchAll">
            <add input="{REQUEST_URI}" pattern="^/api/.*" negate="true" />
            <add input="{REQUEST_FILENAME}" matchType="IsFile" negate="true" />
            <add input="{REQUEST_FILENAME}" matchType="IsDirectory" negate="true" />
        </conditions>
        <action type="Rewrite" url="/" />
    </rule>
</rules>

The problem is, when googlebot tries to access my site, most of the time it comes back with a URL Rewrite error 500. If I remove googlebot from the prerender rule, it will stop throwing the rewrite error, which leads me to believe the issue is with that rule, but their support team tell me it's not.

Can anyone see any glaringly obvious issues with my rules? Or know something I am missing?

prerender googlebot url-rewriting
1 Answers

It turns out this was due to ARR proxy not being turned on. Because I host on Azure, it had to create a new file called applicationHost.xdt and add this:

<?xml version="1.0"?>  
<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">  
    <system.webServer>  
        <proxy xdt:Transform="InsertIfMissing" enabled="true" preserveHostHeader="false"  
        reverseRewriteHostInResponseHeaders="false" />
        <rewrite>
            <allowedServerVariables>
                <add name="HTTP_X_PRERENDER_TOKEN" xdt:Transform="InsertIfMissing" />
            </allowedServerVariables>
        </rewrite>  
    </system.webServer>  
</configuration>

If you use IIS, you can follow this link:

https://learn.microsoft.com/en-us/iis/extensions/configuring-application-request-routing-arr/creating-a-forward-proxy-using-application-request-routing

r3plica0
Related