curl + https://mail.google.com/mail/feed/atom = 401 Unauthorized

user_1243690 0
Viewed 1278

In the past I have successfully been able to access my Gmail account using curl in bash or zsh shell scripts, like this:

USERNAME='example@gmail.com'
PASSWORD='superSeckret'
URL='https://mail.google.com/mail/feed/atom'

curl -s -u "$USERNAME:$PASSWORD" "$URL"

However, now when I attempt this, I get this error:

<HTML>
<HEAD>
<TITLE>Unauthorized</TITLE>
</HEAD>
<BODY BGCOLOR="#FFFFFF" TEXT="#000000">
<H1>Unauthorized</H1>
<H2>Error 401</H2>
</BODY>
</HTML>

If I open https://mail.google.com/mail/feed/atom in a web browser, it shows the feed as expected, but I cannot get it to work via curl.

I tried looking to see if Google had shut down access to this, but I did not see anything that seemed relevant, but I still can't get it to work.

Is there another way to try this?

At minimum what I would like is to get a count of unread messages in my Gmail inbox.

Being able to see the From: and Subject: lines would also be nice.

(I have tried this both from my regular Gmail account as well as my work email which is a Google Suite/Google Apps address which has support for 'less secure apps' enabled. I am actually trying to do this for my work email.)

Any help would be appreciated.

gmail zsh curl bash
4 Answers

after some testing, it appears that to use this endpoint, you first have to try to log in, you will fail, then you must login to gmail normally, then go to https://myaccount.google.com/notifications and find your failed login attempt on the list, and cofirm that was me , after that the https://mail.google.com/mail/feed/atom endpoint should work again with just HTTP Basic Auth.. at least that's what worked for me

user_10670030

After a lot of trial & error, I have found a way that works.

I don't know why this way works and the other way (which used to work) doesn't.

I just know that as of 2021-03-03 this way works and the other doesn't.

For Gmail

If your Gmail address is example@gmail.com then this will work

curl -sfLS "https://example:supersekretpassword@mail.google.com/mail/feed/atom"

For Domains Hosted by Google

If your Google Suite/Google Apps/whatever account is me@example.com then this will work

curl -sfLS "https://me%40example.com:supersekretpassword@mail.google.com/mail/feed/atom"
user_12436900

The option "Less secure app access"

As soon as the option "Less secure app access" is turned off, you get a HTTP status 401 even with the correct credentials on the atom feed endpoint.

Go to https://myaccount.google.com/security to activate the option "Less secure app access" to make sure authentication succeeds again.

This option is automatically set to off after some time when not used, according to https://support.google.com/accounts/answer/6010255?p=less-secure-apps

Using OAuth to authenticate

The smarter way to prevent those authentication issues would be to use OAuth protocol to authenticate: https://developers.google.com/gmail/gmail_inbox_feed

user_71736550

This is how I use OAuth 2.0 for "https://mail.google.com/mail/feed/atom"

Obtain "Client ID"

  1. Create a project in https://console.developers.google.com/
  2. Add "Test User" if you don't want to publish this app
  3. Create OAuth 2.0 Client ID in that project
  4. Obtain the client_id and client_secret

Obtain "Authorization Code"

  1. Replace [client_id] in this below URL with the obtained client_id and visit the link in a browser
 https://accounts.google.com/o/oauth2/auth?client_id=[client_id]&redirect_uri=urn:ietf:wg:oauth:2.0:oob&scope=https://mail.google.com/mail/feed/atom&response_type=code
  1. Obtain the authorization_code to get access_token

Get Access Token

curl --request POST --data "code=[authorization_code]&client_id=[client_id]&client_secret=[client_secret]&redirect_uri=urn:ietf:wg:oauth:2.0:oob&grant_type=authorization_code
  1. Obtain "access_token" and "refresh_token"

Use access_token to access the mail atom feed

curl -H "Authorization: OAuth [access_token]" "https://mail.google.com/mail/feed/atom"

Note

  • if access_token is expired (1 hour), use refresh_token to get a new access_token
curl --request POST --data "--data '&client_id=[client_id]&client_secret=[client_secret]&refresh_token=[refresh_token]&grant_type=refresh_token" https://accounts.google.com/o/oauth2/token
user_80987070
Related