https://developers.google.com/+/web/snippet/ Is this a crawler bot

user_3431477 0
Viewed 4073

I see significant traffic coming from this Bot when we send SMS to users. I am trying to get more details about this bot. Any pointers describing what it does, how important it is, and can we block it would be appreciated. The complete user agent is

Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36 Google (+https://developers.google.com/+/web/snippet/)

googlebot bots
3 Answers

Google's SMS app on Android uses this user agent when it crawls the shared URL to generate a preview inside the chat. This is usually a good thing because your customers will see the preview image and page title, rather than just a link. So I would not recommend blocking it.

user_3454430

You can block google bots on your nginx configuration. Inside of the server{} section, add this:

if ($http_user_agent ~* "developers\.google\.com/\+/web/snippet") {
    return 403;
}
user_130363340

This bot is indeed used by some SMS Android applications to fetch a preview of the we page.

Phones using it include (but it has changed over time) Xiaomi, Huawei, Oppo.. For some websites, it is important to block it. For instance, my app is sending a confirmation link, having it clicked by the preview is messing up our stats and follow-up of who wants that.

To block it, you can use the folowwing .htaccess rule (if you use Apache) :

RewriteEngine on

RewriteCond %{HTTP_USER_AGENT} ^.developers.google.com/+/web/snippet.$ [NC]

RewriteRule "^.*$" - [F,L]

user_4252040
Related