Hive 2.0 log4j vulnerability patching issue

user_4189268 0
Viewed 1040

I have a play around project which uses Hive 2.0.0 and Hadoop 2.6.0, and I was looking into resolving the recent log4j vulnerabilities.

A few of the options would be:

  • Upgrade Hive to the latest version which resolves the vulnerability - this is going to be a complex change with a lot of breaking code changes.
  • Patch Hive 2.0.0 to use latest log4j jars (2.17.1) which resolves the vulnerabilities.
  • Remove JndiLookup.class and other config settings to remediate immediately - which I was able to successfully do.

Hive 2.0.0 uses log4j version 2.4.1 and I am currently trying to patch Hive libs by replacing the 2.4.1 jars with 2.17.1 jars Currently:

./lib/ | grep log4j
log4j-1.2-api-2.4.1.jar
log4j-api-2.4.1.jar
log4j-core-2.4.1.jar
log4j-slf4j-impl-2.4.1.jar
log4j-web-2.4.1.jar

Post patch:

./lib/ | grep log4j
log4j-1.2-api-2.17.1.jar
log4j-api-2.17.1.jar
log4j-core-2.17.1.jar
log4j-slf4j-impl-2.17.1.jar
log4j-web-2.17.1.jar

However, the Hive service (cli/hiveserver2) doesn't start post the jar change and breaks during startup with the following error:

ERROR StatusLogger Unrecognized format specifier [thread]
ERROR StatusLogger Unrecognized conversion specifier [thread] starting at position 25 in conversion pattern.
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/lib/hive-jdbc-2.0.0-standalone.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Exception in thread "main" java.lang.NoSuchMethodError: org.apache.logging.log4j.Logger.debug(Ljava/lang/String;Ljava/lang/Object;)V
    at org.apache.logging.log4j.core.config.yaml.YamlConfigurationFactory.<init>(YamlConfigurationFactory.java:48)
    at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
    ....
    at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
    at org.apache.logging.log4j.core.util.ReflectionUtil.instantiate(ReflectionUtil.java:188)
    at org.apache.logging.log4j.core.config.ConfigurationFactory.addFactory(ConfigurationFactory.java:183)
    at org.apache.logging.log4j.core.config.ConfigurationFactory.getInstance(ConfigurationFactory.java:157)
    at org.apache.logging.log4j.core.LoggerContext.reconfigure(LoggerContext.java:519)
    ....
    at org.apache.logging.log4j.core.impl.Log4jContextFactory.getContext(Log4jContextFactory.java:221)
    at org.apache.logging.log4j.core.config.Configurator.initialize(Configurator.java:140)
    ....
    at org.apache.hadoop.hive.common.LogUtils.initHiveLog4jDefault(LogUtils.java:121)
    at org.apache.hadoop.hive.common.LogUtils.initHiveLog4jCommon(LogUtils.java:73)
    at org.apache.hadoop.hive.common.LogUtils.initHiveLog4j(LogUtils.java:54)
    at org.apache.hive.service.server.HiveServer2.main(HiveServer2.java:631)

Any suggestions on what could be going wrong? Or even such a fix by replacing the jar with the latest version is even viable?

Thank you!!

slf4j hive hadoop log4j
1 Answers

STOP, you can't just replace a jar and hope the newer version will work. This might work on a minor version change. (2.4.X) But most certainly won't work on a major version change (2.X)

Do this (below) then replace the previous jar.

Follow the guidance from log4j:

Otherwise, in any release other than 2.16.0, you may remove the JndiLookup class from the classpath: zip -q -d log4j-core-*.jar org/apache/logging/log4j/core/lookup/JndiLookup.class

Basically it's saying unzip the jar, delete the class file, zip it back up and you can use it. If you did this you are good. You will need to wait for a new version of Hive from apache to get the upgraded version of log4j.

user_135351200
Related