Using JsonLayout with SOLR logs

Viewed 488

I'm trying to log all events from SOLR in JSON format by using JSONLayout. I'm using the official docker image from SOLR but I can't get it to work. Here's my setup:

Dockerfile

FROM solr:7
COPY log4j2-json.xml /opt/solr/server/resources/log4j2.xml

USER root
RUN chmod 644 /opt/solr/server/resources/log4j2.xml
RUN chown solr:solr /opt/solr/server/resources/log4j2.xml

USER solr
EXPOSE 8983
CMD ["solr-precreate", "gettingstarted"]

log4j2.xml original file (without JsonLayout) which does create the solr.log file (using PatternLayout):

<?xml version="1.0" encoding="UTF-8"?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
  -->

<Configuration>
  <Appenders>

    <Console name="STDOUT" target="SYSTEM_OUT">
      <PatternLayout>
        <Pattern>
          %d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard} %X{replica} %X{core}] %c{1.} %m%n
        </Pattern>
      </PatternLayout>
    </Console>

    <RollingFile
        name="RollingFile"
        fileName="${sys:solr.log.dir}/solr.log"
        filePattern="${sys:solr.log.dir}/solr.log.%i" >
      <PatternLayout>
        <Pattern>
          %d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard} %X{replica} %X{core}] %c{1.} %m%n
        </Pattern>
      </PatternLayout>
      <Policies>
        <OnStartupTriggeringPolicy />
        <SizeBasedTriggeringPolicy size="32 MB"/>
      </Policies>
      <DefaultRolloverStrategy max="10"/>
    </RollingFile>

    <RollingFile
        name="SlowFile"
        fileName="${sys:solr.log.dir}/solr_slow_requests.log"
        filePattern="${sys:solr.log.dir}/solr_slow_requests.log.%i" >
      <PatternLayout>
        <Pattern>
          %d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard} %X{replica} %X{core}] %c{1.} %m%n
        </Pattern>
      </PatternLayout>
      <Policies>
        <OnStartupTriggeringPolicy />
        <SizeBasedTriggeringPolicy size="32 MB"/>
      </Policies>
      <DefaultRolloverStrategy max="10"/>
    </RollingFile>

  </Appenders>
  <Loggers>
    <Logger name="org.apache.hadoop" level="warn"/>
    <Logger name="org.apache.solr.update.LoggingInfoStream" level="off"/>
    <Logger name="org.apache.zookeeper" level="warn"/>
    <Logger name="org.apache.solr.core.SolrCore.SlowRequest" level="info" additivity="false">
      <AppenderRef ref="SlowFile"/>
    </Logger>

    <Root level="info">
      <AppenderRef ref="RollingFile"/>
      <AppenderRef ref="STDOUT"/>
    </Root>
  </Loggers>
</Configuration>

log4j2.xml with JsonLayout in RollingFile appender which breaks it. No solr.log file is created:

<?xml version="1.0" encoding="UTF-8"?>
<!--
  Licensed to the Apache Software Foundation (ASF) under one or more
  contributor license agreements.  See the NOTICE file distributed with
  this work for additional information regarding copyright ownership.
  The ASF licenses this file to You under the Apache License, Version 2.0
  (the "License"); you may not use this file except in compliance with
  the License.  You may obtain a copy of the License at

      http://www.apache.org/licenses/LICENSE-2.0

  Unless required by applicable law or agreed to in writing, software
  distributed under the License is distributed on an "AS IS" BASIS,
  WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
  See the License for the specific language governing permissions and
  limitations under the License.
  -->

<Configuration>
  <Appenders>

    <Console name="STDOUT" target="SYSTEM_OUT">
      <PatternLayout>
        <Pattern>
          %d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard} %X{replica} %X{core}] %c{1.} %m%n
        </Pattern>
      </PatternLayout>
    </Console>

    <RollingFile
        name="RollingFile"
        fileName="${sys:solr.log.dir}/solr.log"
        filePattern="${sys:solr.log.dir}/solr.log.%i" >
      <JsonLayout complete="false" compact="true" eventEol="true" />
      <Policies>
        <OnStartupTriggeringPolicy />
        <SizeBasedTriggeringPolicy size="32 MB"/>
      </Policies>
      <DefaultRolloverStrategy max="10"/>
    </RollingFile>

    <RollingFile
        name="SlowFile"
        fileName="${sys:solr.log.dir}/solr_slow_requests.log"
        filePattern="${sys:solr.log.dir}/solr_slow_requests.log.%i" >
      <PatternLayout>
        <Pattern>
          %d{yyyy-MM-dd HH:mm:ss.SSS} %-5p (%t) [%X{collection} %X{shard} %X{replica} %X{core}] %c{1.} %m%n
        </Pattern>
      </PatternLayout>
      <Policies>
        <OnStartupTriggeringPolicy />
        <SizeBasedTriggeringPolicy size="32 MB"/>
      </Policies>
      <DefaultRolloverStrategy max="10"/>
    </RollingFile>

  </Appenders>
  <Loggers>
    <Logger name="org.apache.hadoop" level="warn"/>
    <Logger name="org.apache.solr.update.LoggingInfoStream" level="off"/>
    <Logger name="org.apache.zookeeper" level="warn"/>
    <Logger name="org.apache.solr.core.SolrCore.SlowRequest" level="info" additivity="false">
      <AppenderRef ref="SlowFile"/>
    </Logger>

    <Root level="debug">
      <AppenderRef ref="RollingFile"/>
      <AppenderRef ref="STDOUT"/>
    </Root>
  </Loggers>
</Configuration>

I would expect to see the solr.log file to be created in the logdir and see the events in json format. But the solr.log file isn't even created. SOLR does however start.

What am I missing?

Thanks for your help.

PS. I found this thread which describes the same problem but no solution: see here

1 Answers

On solr official docker image 8.11.2 the same problem is caused by a missing jackson library in the classpath:

ERROR Could not create plugin of type class org.apache.logging.log4j.core.layout.JsonLayout for element JsonLayout: java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ser/FilterProvider java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ser/FilterProvider
at org.apache.logging.log4j.core.layout.JsonLayout.<init>(JsonLayout.java:159)

This is definitely a bug (https://issues.apache.org/jira/browse/SOLR-16342), but since the jackson library exists elsewhere in the original image, there is a possible workaround to apply while building the docker. Basically you can link the needed jackson jars that exists in /opt/solr/server/solr-webapp/webapp/WEB-INF/lib/ to /opt/solr/server/lib/ext/:

RUN ln -s /opt/solr/server/solr-webapp/webapp/WEB-INF/lib/jackson-core-2.13.3.jar /opt/solr/server/lib/ext/jackson-core-2.13.3.jar && \
ln -s /opt/solr/server/solr-webapp/webapp/WEB-INF/lib/jackson-databind-2.13.3.jar /opt/solr/server/lib/ext/jackson-databind-2.13.3.jar && \
ln -s /opt/solr/server/solr-webapp/webapp/WEB-INF/lib/jackson-annotations-2.13.3.jar /opt/solr/server/lib/ext/jackson-annotations-2.13.3.jar

Moreover:

JsonTemplate is considered deprecated. JsonTemplateLayout provides more capabilitites and should be used instead.

(see https://logging.apache.org/log4j/2.x/manual/layouts.html#JSONLayout)

If you move to <JsonTemplateLayout /> it works out of the box in solr.

Related