How to setup a distributed cluster in keycloak 17

user_7870307 0
Viewed 1154

When keycloak was running on wildfly we could specify cluster configuration in standalone-ha.xml using jgroups

How can we do the same in quarkus distribution for keycloak 17?

I have tried setting up the property given below in keycloak.conf

cache-config-file=conf/cache-ispn.xml

And succeeded in setting up a cluster when two keycloak instances are running on the same machine.

However, if two keycloak instances are running in different servers (distributed cluster), what are the configurations needed for the servers to discover each other?

quarkus keycloak
2 Answers

There are a few options when setting up a cluster.

As noted here: https://www.keycloak.org/2019/05/keycloak-cluster-setup.html , you can use the following:

  1. PING
  2. TCPPING
  3. JDBC_PING
  4. On Kubernetes we can use DNS_PING or KUBE_PING

Thanks to: https://gist.github.com/xgp/768eea11f92806b9c83f95902f7f8f80 I have found a solution that uses JDBC_PING.

Have a stack in your configuration that looks like this:

<jgroups>
<stack name="jdbc-ping-tcp" extends="tcp">
  <TCP external_addr="${env.JGROUPS_DISCOVERY_EXTERNAL_IP}" bind_port="7600" />
  <JDBC_PING connection_driver="org.postgresql.Driver"
             connection_username="${env.KC_DB_USERNAME}" connection_password="${env.KC_DB_PASSWORD}"
             connection_url="jdbc:postgresql://${env.KC_DB_URL_HOST}/${env.KC_DB_URL_DATABASE}"
             initialize_sql="CREATE SCHEMA IF NOT EXISTS ${env.KC_DB_SCHEMA:public}; CREATE TABLE IF NOT EXISTS ${env.KC_DB_SCHEMA:public}.JGROUPSPING (own_addr varchar(200) NOT NULL, cluster_name varchar(200) NOT NULL, bind_addr varchar(200) NOT NULL, updated timestamp default current_timestamp, ping_data BYTEA, constraint PK_JGROUPSPING PRIMARY KEY (own_addr, cluster_name));"
             insert_single_sql="INSERT INTO ${env.KC_DB_SCHEMA:public}.JGROUPSPING (own_addr, cluster_name, bind_addr, updated, ping_data) values (?, ?, '${env.JGROUPS_DISCOVERY_EXTERNAL_IP:127.0.0.1}', NOW(), ?);"
             delete_single_sql="DELETE FROM ${env.KC_DB_SCHEMA:public}.JGROUPSPING WHERE own_addr=? AND cluster_name=?;"
             select_all_pingdata_sql="SELECT ping_data, own_addr, cluster_name FROM ${env.KC_DB_SCHEMA:public}.JGROUPSPING WHERE cluster_name=?"
             info_writer_sleep_time="500"
             remove_all_data_on_view_change="true"
             stack.combine="REPLACE"
             stack.position="MPING" />
</stack>
</jgroups>

Please note that you'll need to make sure that the environment variables used here are defined:

  1. KC_DB_URL_HOST
  2. KC_DB_URL_DATABASE
  3. KC_DB_SCHEMA
  4. KC_DB_USERNAME
  5. KC_DB_PASSWORD
  6. JGROUPS_DISCOVERY_EXTERNAL_IP

NB:

  • JGROUPS_DISCOVERY_EXTERNAL_IP is very important
  • The port used here is 7600, change as needed, but make sure its exposed if you're using docker
user_60767610

I found a way to use TCPPING too.

posting the 'cache-ispn.xml' config that worked for me. I have one node running on port 7605 and the other on port 7805. The bind_addr and bind_port of the node can be defined in 'TCP' element the hosts in the cluster can be defined in 'initial_hosts' attribute of 'TCPPING' element

Important - 'cache-stack' should not be defined in keycloak.conf , else it will override the jgroups stack you define in 'cache-ispn.xml'

        <jgroups>                                                                                                                        
        <stack name="tcpping" extends="tcp">
          <TCP bind_addr="127.0.0.1" bind_port="7605"/>                                                                                                                         
          <TCPPING                                                                                                                                                                                                                                            
            initial_hosts="127.0.0.1[7605],127.0.0.1[7805]"                                                                                                                        
            port_range="0"                                                                                                                                                                                                                                               
            stack.combine="REPLACE"                                                                                                                        
            stack.position="MPING"                                                                                                                        
            />                                                                                                                        
        </stack>                                                                                                                        
      </jgroups>   

<cache-container name="keycloak">
    <transport cluster="testcluster" node-name="test-node" stack="tcpping" lock-timeout="60000"/>
user_78703070
Related