HBase masters are going down after enabling Ranger plugin

user_8164424 0
Viewed 398

I have a 3 node Distributed Hbase cluster of version 2.0.2 and its working properly. After I Installed apache ranger From Same Ambari & I just Enabled the Hbase plugin and then I restarted the HBase Immediately Both HBase masters are getting down with Below logs.

 ERROR [Thread-16] master.HMaster: ***** ABORTING master ,16000,1585061451214: The coprocessor org.apache.ranger.authorization.hbase.RangerAuthorizationCoprocessor threw java.lang.UnsatisfiedLinkError: /run/hbase/.cache/JNA/temp/jna2781046120401699711.tmp: /run/hbase/.cache/JNA/temp/jna2781046120401699711.tmp: failed to map segment from shared object *****


ERROR [Thread-16] master.HMaster: Failed to become active master
java.lang.NullPointerException
    at org.apache.hadoop.hbase.master.HMaster.startProcedureExecutor(HMaster.java:1405)
    at org.apache.hadoop.hbase.master.HMaster.startServiceThreads(HMaster.java:1310)
    at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:930)
    at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2234)
    at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:570)
    at java.lang.Thread.run(Thread.java:745)

If I Disable the Plugin and everything is working fine

ambari apache-ranger hbase hadoop
1 Answers

We were experiencing a similar issue with a near identical stack trace to yours. Like you, everything worked only if the plugin was disabled.

For us it turns out that the reason was that the /run mount had a noexec flag, which prevented tmp files from being executed within it. The solution was to remount /run on the Hbase master nodes using:

sudo mount -o remount,exec /run

After this we restarted the HBase services and everything was working again.

Note that this modified mount will not be saved after a restart of the machine. For it to persist you may need to use something like fstab

user_139373410
Related