Use azure active directory with NEST/Elasticsearch.net

Viewed 406

We are working on a project where we are authenticating users with Azure Active Directory. Upon the successful authentication, the user's browsers receive an Id and Access token, and then we use the same access token to query other Microsoft products (Sharepoint, OneDrive, etc).

We are planning to use Elastic Search for our search need. We have already set up SAML/ OpenId realms on our ECE Deployment Portal and Cluster. So if any users try to access ECE deployment portal/ Kibana, they will be prompted to authenticate against Microsoft Azure AD, and upon successful authentication, they get redirected to ECE or Kibana.

We are using C# and NEST dll (ElasticSearch.Net) to create queries and search the elastic search end point. We are not sure how exactly should we use the access token received on the UI side with Elastic Search to query out indices. We know, we can use native user credentials or API keys to access the elastic search but we want to use the same azure ad authentication flow(SAML/OpenID) to access Elastic Search.

Is it possible to use the Azure AD access token received on the UI side to access & query Elastic Search Clusters or is there any other way to re-authenticate users while they try to access Elastic Search Cluster? Is there a way to authenticate the users with elastic search end point and generate an access token that can be used to query elastic search further?

In short, we want to re-authenticate users with Elastic Search while querying the data?

var settings = new ConnectionSettings(new Uri(mEsQuerySource.Url));
settings.BasicAuthentication("user", "plain text password");
mClient = new ElasticClient(settings);
1 Answers

Thank You Tim for sharing the solution over elastic Portal. I am updating the same answer over here to help other community member.

In current versions of Elasticsearch (as I write this, 7.14 is the latest version) there is no way to use an Azure AD access token to directly access Elasticsearch.
That is, you cannot have your application authenticate directly to AAD and then use the tokens you receive from AAD as a credential to authenticate to Elasticsearch.
There is no authentication provider in Elasticsearch that works with arbitrary tokens from an external issuer.

You can however do the same thing that ECE and Kibana do and perform SAML or OpenID Connect authentication via Elasticsearch, in order to generate Elasticsearch access & refresh tokens (which are separate from the Azure AD tokens).

There is documentation on how to perform SAML 3 or OIDC 2 authentication to Elasticsearch via a custom application.

The high-level overview would be (I assume SAML here, but OIDC would be similar):

When a user accesses your application they would authenticate against AzureAD as normal

Then, you would use the Elasticsearch APIs to perform an additional authentication against an Elasticsearch SAML realm with Elasticsearch as the service provider and AzureAD as the Identity Provider. Since the user is already authenticated within Azure AD, that second authentication process should be transparent to the user - AAD will simply issue a new SAML assertion with Elasticsearch as the recipient.

Those Elasticsearch APIs will accept the SAML assertion, and return a pair of tokens (access + refresh) that can be used to authenticate to Elasticsearch

Your application will retain the access + refresh tokens for the user's session

The access token will be used to authenticate when accessing Elasticsearch APIs

The refresh token will be used to generate a new access token when the old one expires (or is about to expire).

If your users are in an identity store that Elasticsearch can query (e.g. something that supports LDAP search), then another option is to use the Elasticsearch run-as capability.
In this case your application would authenticate to Elasticsearch using a single system credential (probably a user in the native realm). That user would have permission to run-as all other users and this can be used to perform searches on behalf of your end users without needing them to authenticate directly to Elasticsearch.

The final option would be to implement a custom realm, if you have engineers who are comfortable writing Java

Reference: Use azure active directory with NEST/Elasticsearch.net - Elastic Stack / Elasticsearch - Discuss the Elastic Stack

Related