How to do a JUnit assert on a message in a logger

Here is a simple and efficient Logback solution.
It doesn't require to add/create any new class.
It relies on ListAppender : a whitebox logback appender where log entries are added in a public List field that we could so use to make our assertions.

Here is a simple example.

Foo class :

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

public class Foo {

    static final Logger LOGGER = LoggerFactory.getLogger(Foo .class);

    public void doThat() {
        LOGGER.info("start");
        //...
        LOGGER.info("finish");
    }
}

FooTest class :

import org.slf4j.LoggerFactory;
import ch.qos.logback.classic.Level;
import ch.qos.logback.classic.Logger;
import ch.qos.logback.classic.spi.ILoggingEvent;
import ch.qos.logback.core.read.ListAppender;

public class FooTest {

    @Test
    void doThat() throws Exception {
        // get Logback Logger 
        Logger fooLogger = (Logger) LoggerFactory.getLogger(Foo.class);

        // create and start a ListAppender
        ListAppender<ILoggingEvent> listAppender = new ListAppender<>();
        listAppender.start();

        // add the appender to the logger
        // addAppender is outdated now
        fooLogger.addAppender(listAppender);

        // call method under test
        Foo foo = new Foo();
        foo.doThat();

        // JUnit assertions
        List<ILoggingEvent> logsList = listAppender.list;
        assertEquals("start", logsList.get(0)
                                      .getMessage());
        assertEquals(Level.INFO, logsList.get(0)
                                         .getLevel());

        assertEquals("finish", logsList.get(1)
                                       .getMessage());
        assertEquals(Level.INFO, logsList.get(1)
                                         .getLevel());
    }
}

JUnit assertions don't sound very adapted to assert some specific properties of the list elements.
Matcher/assertion libraries as AssertJ or Hamcrest appears better for that :

With AssertJ it would be :

import org.assertj.core.api.Assertions;

Assertions.assertThat(listAppender.list)
          .extracting(ILoggingEvent::getMessage, ILoggingEvent::getLevel)
          .containsExactly(Tuple.tuple("start", Level.INFO), Tuple.tuple("finish", Level.INFO));

For Junit 5 (Jupiter) Spring's OutputCaptureExtension is quite useful. Its available since Spring Boot 2.2 and is available in the spring-boot-test artifact.

Example (taken from javadoc):

@ExtendWith(OutputCaptureExtension.class)
class MyTest {
    @Test
    void test(CapturedOutput output) {
        System.out.println("ok");
        assertThat(output).contains("ok");
        System.err.println("error");
    }

    @AfterEach
    void after(CapturedOutput output) {
        assertThat(output.getOut()).contains("ok");
        assertThat(output.getErr()).contains("error");
    }
}

I also ran into the same challanged and ended up at this page. Although I am 11 years too late to answers the question, I thought maybe it could be still usefull for others. I found the answer of davidxxx with Logback and the ListAppander very usefull. I used the same configuration for multiple projects, however it was not so fun to copy/paste it and maintaining all the version when I needed to changes something. I thought it would be better to make a library out of it and contribute back to the community. It works with SLFJ4, Log4j, Log4j2, Java Util Logging, JBoss Logging and with Lombok annotations. Please have a look here: LogCaptor for detailed examples and how to add it to your project.

Example situation:

public class FooService {

    private static final Logger LOGGER = LoggerFactory.getLogger(FooService.class);

    public void sayHello() {
        LOGGER.warn("Congratulations, you are pregnant!");
    }

}

Example unit test with usage of LogCaptor:

import nl.altindag.log.LogCaptor;
import org.junit.jupiter.api.Test;

import static org.assertj.core.api.Assertions.assertThat;

public class FooServiceTest {

    @Test
    public void sayHelloShouldLogWarnMessage() {
        LogCaptor logCaptor = LogCaptor.forClass(FooService.class);

        FooService fooService = new FooService();
        fooService.sayHello();

        assertThat(logCaptor.getWarnLogs())
            .contains("Congratulations, you are pregnant!");
    }
}

I wasn't quite sure if I should post this here, because it could also be seen as a way to promote "my library" but I thought it could be helpful for developers who have the same challenges.

Wow. I'm unsure why this was so hard. I found I was unable to use any of the code samples above because I was using log4j2 over slf4j. This is my solution:

public class SpecialLogServiceTest {

  @Mock
  private Appender appender;

  @Captor
  private ArgumentCaptor<LogEvent> captor;

  @InjectMocks
  private SpecialLogService specialLogService;

  private LoggerConfig loggerConfig;

  @Before
  public void setUp() {
    // prepare the appender so Log4j likes it
    when(appender.getName()).thenReturn("MockAppender");
    when(appender.isStarted()).thenReturn(true);
    when(appender.isStopped()).thenReturn(false);

    final LoggerContext ctx = (LoggerContext) LogManager.getContext(false);
    final Configuration config = ctx.getConfiguration();
    loggerConfig = config.getLoggerConfig("org.example.SpecialLogService");
    loggerConfig.addAppender(appender, AuditLogCRUDService.LEVEL_AUDIT, null);
  }

  @After
  public void tearDown() {
    loggerConfig.removeAppender("MockAppender");
  }

  @Test
  public void writeLog_shouldCreateCorrectLogMessage() throws Exception {
    SpecialLog specialLog = new SpecialLogBuilder().build();
    String expectedLog = "this is my log message";

    specialLogService.writeLog(specialLog);

    verify(appender).append(captor.capture());
    assertThat(captor.getAllValues().size(), is(1));
    assertThat(captor.getAllValues().get(0).getMessage().toString(), is(expectedLog));
  }
}

Here is the sample code to mock log, irrespective of the version used for junit or sping, springboot.

import ch.qos.logback.classic.spi.LoggingEvent;
import ch.qos.logback.core.Appender;
import org.mockito.ArgumentMatcher;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.junit.Test;

import static org.mockito.Matchers.argThat;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.verify;
import static org.mockito.Mockito.when;

public class MyTest {
  private static Logger logger = LoggerFactory.getLogger(MyTest.class);

    @Test
    public void testSomething() {
    ch.qos.logback.classic.Logger root = (ch.qos.logback.classic.Logger) LoggerFactory.getLogger(ch.qos.logback.classic.Logger.ROOT_LOGGER_NAME);
    final Appender mockAppender = mock(Appender.class);
    when(mockAppender.getName()).thenReturn("MOCK");
    root.addAppender(mockAppender);

    //... do whatever you need to trigger the log

    verify(mockAppender).doAppend(argThat(new ArgumentMatcher() {
      @Override
      public boolean matches(final Object argument) {
        return ((LoggingEvent)argument).getFormattedMessage().contains("Hey this is the message I want to see");
      }
    }));
  }
}

Note that in Log4J 2.x, the public interface org.apache.logging.log4j.Logger doesn't include the setAppender() and removeAppender() methods.

But if you're not doing anything too fancy, you should be able to cast it to the implementation class org.apache.logging.log4j.core.Logger, which does expose those methods.

Here's an example with Mockito and AssertJ:

// Import the implementation class rather than the API interface
import org.apache.logging.log4j.core.Logger;

// Cast logger to implementation class to get access to setAppender/removeAppender
Logger log = (Logger) LogManager.getLogger(MyClassUnderTest.class);

// Set up the mock appender, stubbing some methods Log4J needs internally
Appender appender = mock(Appender.class);
when(appender.getName()).thenReturn("Mock Appender");
when(appender.isStarted()).thenReturn(true);

log.addAppender(appender);
try {
    new MyClassUnderTest().doSomethingThatShouldLogAnError();
} finally {
    log.removeAppender(appender);
}

// Verify that we got an error with the expected message
ArgumentCaptor<LogEvent> logEventCaptor = ArgumentCaptor.forClass(LogEvent.class);
verify(appender).append(logEventCaptor.capture());
LogEvent logEvent = logEventCaptor.getValue();
assertThat(logEvent.getLevel()).isEqualTo(Level.ERROR);
assertThat(logEvent.getMessage().getFormattedMessage()).contains(expectedErrorMessage);

You don't need to rely on hardcoded static global Loggers in your class implementation, you can provide a default logger in the default constructor and then use a specific constructor to set a reference to the provided logger.

class MyClassToTest {
    private final Logger logger;
    
    public MyClassToTest() {
      this(SomeStatic.logger);
    };
    
    MyClassToTest(Logger logger) {
      this.logger = logger;
    };
    
    public void someOperation() {
        logger.warn("warning message");
        // ...
    };
};

class MyClassToTestTest {
    
    @Test
    public warnCalled() {
        Logger loggerMock = mock(Logger.class);
        MyClassTest myClassToTest = new MyClassToTest(logger);
        myClassToTest.someOperation();
        verify(loggerMock).warn(anyString());
    };
}

Check this library https://github.com/Hakky54/log-captor

Include in your maven file the reference for the library:

<dependency>
    <groupId>io.github.hakky54</groupId>
    <artifactId>logcaptor</artifactId>
    <version>2.5.0</version>
    <scope>test</scope>
</dependency>

In java code test method you should include this:

LogCaptor logCaptor = LogCaptor.forClass(MyClass.class);

 // do the test logic....

assertThat(logCaptor.getLogs()).contains("Some log to assert");

Easiest way

  @ExtendWith(OutputCaptureExtension.class)
  class MyTestClass { 
    
          @Test
          void my_test_method(CapturedOutput output) {
               assertThat(output).contains("my test log.");
          }
  }

The API for Log4J2 is slightly different. Also you might be using its async appender. I created a latched appender for this:

    public static class LatchedAppender extends AbstractAppender implements AutoCloseable {

    private final List<LogEvent> messages = new ArrayList<>();
    private final CountDownLatch latch;
    private final LoggerConfig loggerConfig;

    public LatchedAppender(Class<?> classThatLogs, int expectedMessages) {
        this(classThatLogs, null, null, expectedMessages);
    }
    public LatchedAppender(Class<?> classThatLogs, Filter filter, Layout<? extends Serializable> layout, int expectedMessages) {
        super(classThatLogs.getName()+"."+"LatchedAppender", filter, layout);
        latch = new CountDownLatch(expectedMessages);
        final LoggerContext ctx = (LoggerContext) LogManager.getContext(false);
        final Configuration config = ctx.getConfiguration();
        loggerConfig = config.getLoggerConfig(LogManager.getLogger(classThatLogs).getName());
        loggerConfig.addAppender(this, Level.ALL, ThresholdFilter.createFilter(Level.ALL, null, null));
        start();
    }

    @Override
    public void append(LogEvent event) {
        messages.add(event);
        latch.countDown();
    }

    public List<LogEvent> awaitMessages() throws InterruptedException {
        assertTrue(latch.await(10, TimeUnit.SECONDS));
        return messages;
    }

    @Override
    public void close() {
        stop();
        loggerConfig.removeAppender(this.getName());
    }
}

Use it like this:

        try (LatchedAppender appender = new LatchedAppender(ClassUnderTest.class, 1)) {

        ClassUnderTest.methodThatLogs();
        List<LogEvent> events = appender.awaitMessages();
        assertEquals(1, events.size());
        //more assertions here

    }//appender removed

I answered a similar question for log4j see how-can-i-test-with-junit-that-a-warning-was-logged-with-log4

This is newer and example with Log4j2 (tested with 2.11.2) and junit 5;

    package com.whatever.log;

    import org.apache.logging.log4j.Level;
    import org.apache.logging.log4j.LogManager;
    import org.apache.logging.log4j.core.Logger;
    import org.apache.logging.log4j.core.*;
    import org.apache.logging.log4j.core.appender.AbstractAppender;
    import org.apache.logging.log4j.core.config.Configuration;
    import org.apache.logging.log4j.core.config.LoggerConfig;
    import org.apache.logging.log4j.core.config.plugins.Plugin;
    import org.apache.logging.log4j.core.config.plugins.PluginAttribute;
    import org.apache.logging.log4j.core.config.plugins.PluginElement;
    import org.apache.logging.log4j.core.config.plugins.PluginFactory;
    import org.junit.jupiter.api.AfterEach;
    import org.junit.jupiter.api.BeforeEach;
    import org.junit.jupiter.api.DisplayName;
    import org.junit.jupiter.api.Test;

    import java.util.ArrayList;
    import java.util.List;
    import static org.junit.Assert.*;

class TestLogger {

    private TestAppender testAppender;
    private LoggerConfig loggerConfig;
    private final Logger logger = (Logger)
            LogManager.getLogger(ClassUnderTest.class);

    @Test
    @DisplayName("Test Log Junit5 and log4j2")
    void test() {
        ClassUnderTest.logMessage();
        final LogEvent loggingEvent = testAppender.events.get(0);
        //asset equals 1 because log level is info, change it to debug and
        //the test will fail
        assertTrue(testAppender.events.size()==1,"Unexpected empty log");
        assertEquals(Level.INFO,loggingEvent.getLevel(),"Unexpected log level");
        assertEquals(loggingEvent.getMessage().toString()
                ,"Hello Test","Unexpected log message");
    }

    @BeforeEach
    private void setup() {
        testAppender = new TestAppender("TestAppender", null);

        final LoggerContext context = logger.getContext();
        final Configuration configuration = context.getConfiguration();

        loggerConfig = configuration.getLoggerConfig(logger.getName());
        loggerConfig.setLevel(Level.INFO);
        loggerConfig.addAppender(testAppender,Level.INFO,null);
        testAppender.start();
        context.updateLoggers();
    }

    @AfterEach
    void after(){
        testAppender.stop();
        loggerConfig.removeAppender("TestAppender");
        final LoggerContext context = logger.getContext();
        context.updateLoggers();
    }

    @Plugin( name = "TestAppender", category = Core.CATEGORY_NAME, elementType = Appender.ELEMENT_TYPE)
    static class TestAppender extends AbstractAppender {

        List<LogEvent> events = new ArrayList();

        protected TestAppender(String name, Filter filter) {
            super(name, filter, null);
        }

        @PluginFactory
        public static TestAppender createAppender(
                @PluginAttribute("name") String name,
                @PluginElement("Filter") Filter filter) {
            return new TestAppender(name, filter);
        }

        @Override
        public void append(LogEvent event) {
            events.add(event);
        }
    }

    static class ClassUnderTest {
        private static final Logger LOGGER =  (Logger) LogManager.getLogger(ClassUnderTest.class);
        public static void logMessage(){
            LOGGER.info("Hello Test");
            LOGGER.debug("Hello Test");
        }
    }
}

Using the following maven dependencies

 <dependency>
 <artifactId>log4j-core</artifactId>
  <packaging>jar</packaging>
  <version>2.11.2</version>
</dependency>
<dependency>
    <groupId>org.junit.jupiter</groupId>
    <artifactId>junit-jupiter-api</artifactId>
    <version>5.5.0</version>
    <scope>test</scope>
</dependency>

In my case I solved the same issue as bellow:

Logger root = (Logger) LoggerFactory.getLogger(CSVTasklet.class); //CSVTasklet is my target class
    final Appender mockAppender = mock(Appender.class);
    root.addAppender(mockAppender); 

verify(mockAppender).doAppend(argThat((ArgumentMatcher) argument -> ((LoggingEvent) argument).getMessage().contains("No projects."))); // I checked "No projects." in the log

Unit testing by adding an Appender does not really test the Logger's configuration. So, I think that it's one of the unique cases in which unit tests do not bring that much value, but an integration test brings a lot of value (especially if your logging has some auditing purposes).

In order to create an integration test for it, let us suppose that you are running with a simple ConsoleAppender and want to test its output. Then, you should test how the message is written to its own ByteArrayOutputStream from System.out.

In that sense, I would do the following (I'm using JUnit 5):

public class Slf4jAuditLoggerTest {

    private final ByteArrayOutputStream outContent = new ByteArrayOutputStream();

    @BeforeEach
    public void beforeEach() {
        System.setOut(new PrintStream(outContent));
    }

In that way, you are able to test its output by simply:

    @Test
    public void myTest() {
        // Given...
        // When...
        // Then
        assertTrue(outContent.toString().contains("[INFO] My formatted string from Logger"));
    }

If you do so, you will bring much more value to your project and will not need to using an in-memory implementation, create a new Appender, or whatsoever.

If you are using log4j2, the solution from https://www.dontpanicblog.co.uk/2018/04/29/test-log4j2-with-junit/ allowed me to assert messages were logged.

The solution goes like this:

• Define a log4j appender as an ExternalResource rule

  public class LogAppenderResource extends ExternalResource {
  
  private static final String APPENDER_NAME = "log4jRuleAppender";
  
  /**
   * Logged messages contains level and message only.
   * This allows us to test that level and message are set.
   */
  private static final String PATTERN = "%-5level %msg";
  
  private Logger logger;
  private Appender appender;
  private final CharArrayWriter outContent = new CharArrayWriter();
  
  public LogAppenderResource(org.apache.logging.log4j.Logger logger) {
      this.logger = (org.apache.logging.log4j.core.Logger)logger;
  }
  
  @Override
  protected void before() {
      StringLayout layout = PatternLayout.newBuilder().withPattern(PATTERN).build();
      appender = WriterAppender.newBuilder()
              .setTarget(outContent)
              .setLayout(layout)
              .setName(APPENDER_NAME).build();
      appender.start();
      logger.addAppender(appender);
  }
  
  @Override
  protected void after() {
      logger.removeAppender(appender);
  }
  
  public String getOutput() {
      return outContent.toString();
      }
  }
  

• Define a test that use your ExternalResource rule

  public class LoggingTextListenerTest {
  
      @Rule public LogAppenderResource appender = new LogAppenderResource(LogManager.getLogger(LoggingTextListener.class)); 
      private LoggingTextListener listener = new LoggingTextListener(); //     Class under test
  
      @Test
      public void startedEvent_isLogged() {
      listener.started();
      assertThat(appender.getOutput(), containsString("started"));
      }
  }
  

Don't forget to have log4j2.xml as part of src/test/resources