Grafana Oauth Proxy still displaying native login form

user_13806973 0
Viewed 1840

Attempting to use Google's Oauth Proxy service and Grafana's Auth Proxy configuration, but Grafana still displays login form.

Google login dialog is displayed as expected, but once authenticated it is expected that the user is then authenticated by Grafana.

Setup: Kubernetes (AWS/EKS) Oauth Proxy enabled for ingress-nginx

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: oauth2-proxy
  namespace: monitoring
spec:
  rules:
  - host: grafana.*domain*
    http:
      paths:
      - backend:
          serviceName: oauth2-proxy
          servicePort: 4180
        path: /oauth2
...
  tls:
  - hosts:
    - grafana.*domain*

Grafana ingress:

...
Rules:
  Host                        Path  Backends
  ----                        ----  --------
  grafana.*domain*  
                              /   prometheus-operator-grafana:80 (192.168.2.91:3000)
Annotations:                  helm.fluxcd.io/antecedent: monitoring:helmrelease/prometheus-operator
                              kubernetes.io/ingress.class: nginx
                              nginx.ingress.kubernetes.io/auth-signin: https://$host/oauth2/start?rd=$escaped_request_uri
                              nginx.ingress.kubernetes.io/auth-url: https://$host/oauth2/auth

Grafana.ini: as per https://grafana.com/docs/grafana/latest/auth/auth-proxy/

[analytics]
check_for_updates = true
[auth]
oauth_auto_login = true
signout_redirect_url = https://grafana.*domain*
[auth.proxy]
auto_sign_up = true
enable_login_token = false
enabled = true
header_name = X-WEBAUTH-USER
header_property = username
headers = Name:X-WEBAUTH-NAME Email:X-WEBAUTH-EMAIL Groups:X-WEBAUTH-GROUPS
[grafana_net]
url = https://grafana.net
[log]
mode = console
[paths]
data = /var/lib/grafana/data
logs = /var/log/grafana
plugins = /var/lib/grafana/plugins
provisioning = /etc/grafana/provisioning
[server]
domain = *domain*
root_url = https://grafana.*domain*
[users]
allow_sign_up = false
auto_assign_org = true
auto_assign_org_role = Admin

User is prompted for Google Authentication as desired. However Grafana still presents login dialog, despite presence of

[auth]
oauth_auto_login = true

Log from Nginx

192.168.77.87 - xxxxxxxxxxxxxxxxx [2020/06/24 15:59:52] [AuthSuccess] Authenticated via OAuth2: Session{email:xxxxxxxxxn@domain user:nnnnnnnnnnnnnnnnnnnn PreferredUsername: token:true id_token:true created:2020-06-24 15:59:52.238393221 +0000 UTC m=+106369.587921725 expires:2020-06-24 16:59:51 +0000 UTC refresh_token:true}
192.168.77.87 - - [2020/06/24 15:59:52] grafana.domain GET - "/oauth2/callback?state=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx:/&code=xxxxxxxxxxxxxxxxxxxxxxxxx&scope=email%20profile%20https://www.googleapis.com/auth/userinfo.profile%20https://www.googleapis.com/auth/userinfo.email%20openid&authuser=0&hd=domain&prompt=consent" HTTP/1.1 "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:77.0) Gecko/20100101 Firefox/77.0" 302 24 0.181

Log from Oauth Proxy:

192.168.17.214 - - [24/Jun/2020:15:59:52 +0000] "GET /oauth2/auth HTTP/1.1" 202 0 "https://accounts.google.com/signin/oauth/oauthchooseaccount?access_type=offline&acr_values&approval_prompt=force&client_id=client_id&redirect_uri=https%3A%2F%2Fgrafana.domain%2Foauth2%2Fcallback&response_type=code&scope=profile%20email&state=xxxxxxxxxxxxxxxxxxxxxxxxxx&flowName=GeneralOAuthFlow" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:77.0) Gecko/20100101 Firefox/77.0" 1217 0.004 [monitoring-oauth2-proxy-4180] [] 192.168.44.224:4180 0 0.004 202 xxxxxxxxxxxxxxxxxxxxxxxxxx
192.168.6.127 - - [24/Jun/2020:15:59:52 +0000] "GET /login HTTP/2.0" 202 0 "https://accounts.google.com/signin/oauth/oauthchooseaccount?access_type=offline&acr_values&approval_prompt=force&client_id=client_id&redirect_uri=https%3A%2F%2Fgrafana.domain%2Foauth2%2Fcallback&response_type=code&scope=profile%20email&state=xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxQg&flowName=GeneralOAuthFlow" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:77.0) Gecko/20100101 Firefox/77.0" 0 0.052 [monitoring-prometheus-operator-grafana-80] [] 54.76.77.91:443 0 0.048 202 xxxxxxxxxxxxxxxxxxxxxxxxxxxx
192.168.6.127 - - [24/Jun/2020:15:59:52 +0000] "GET /login HTTP/2.0" 200 6822 "https://accounts.google.com/signin/oauth/oauthchooseaccount?access_type=offline&acr_values&approval_prompt=force&client_id=client_id&redirect_uri=https%3A%2F%2Fgrafana.dmoain%2Foauth2%2Fcallback&response_type=code&scope=profile%20email&state=xxxxxxxxxxxxxxxxxxxxxxxxx&flowName=GeneralOAuthFlow" "Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:77.0) Gecko/20100101 Firefox/77.0" 32 0.056 [monitoring-prometheus-operator-grafana-80] [] 192.168.2.91:3000 27042 0.008 200 xxxxxxxxxxxxxxxxxxxxxxxxxxx

Note instance can be made to work via auth.google however that requires the secret held with grafana.ini where other clients are secured at ingress-nginx.

Q. What should the Oauth callback be set to for Grafana Oauth Proxy?

grafana kubernetes oauth nginx proxy
1 Answers

So the issue is that although the user is authenticated as far a Google (as OAuth Provider), and Oauth-Proxy is concerned this is not reflected in the user experience as far a Grafana (being the upstream application) is concerned.

So both Oauth-Proxy and Google (the Oauth Provider) were both configured correctly. Namely:

  1. Oauth-Proxy requires --set-xauthrequest as part of the command line options.
  2. Google's redirect url to be https://<host>/oauth2/callback. Note this is different than when using Grafana's built in proxy.

Problem is that the authenticated user's details where either not getting back to Grafana or they weren't being acknowledged. There is a great deal is inadequate documentation on this, partly from Grafana, partly from the Ingress Nginx Controller. This had nothing to do with Oauth-proxy as has been speculated.

My solution is as follows:

  grafana.ini:
    auth:
      oauth_auto_login: true
      signout_redirect_url: "https://<host>/oauth2/sign_out"
    auth.proxy:
      enabled: true
      header_name: X-Email
      header_property: email
      auto_sign_up: true
    users:
      allow_sign_up: false
      auto_assign_org: true
      auto_assign_org_role: Viewer

This is not intended to be complete but it covers the main options regarding authentication and user profiles.

Grafana Ingress also requires the following annotations:

  kubernetes.io/ingress.class: "nginx"
  nginx.ingress.kubernetes.io/auth-url: "https://$host/oauth2/auth"
  nginx.ingress.kubernetes.io/auth-signin: "https://$host/oauth2/start?rd=$escaped_request_uri"
  nginx.ingress.kubernetes.io/configuration-snippet: |
    auth_request_set $user   $upstream_http_x_auth_request_user;
    auth_request_set $email  $upstream_http_x_auth_request_email;
    proxy_set_header X-User  $user;
    proxy_set_header X-Email $email;

There may be a way of specifying the 4 snippet lines with a single annotation, but I was not able to find it.

The above has a number of advantages over Grafana's built in proxy:

  1. It enables multiple applications to be configured with the common Authentication backend. This gives a single source of truth for the secrets.
  2. Bad actors are stopped at the Reverse Proxy, not the application.
  3. This solution works with Prometheus Operator without the need to place secrets within your code, there currently being an issue with the prometheus operator not working correctly with environment variables (set by secrets).
user_138069730
Related