Spring boot application does not find placeholder 'spring.embedded.kafka.brokers' for a spock-spring test

user_1718681 0
Viewed 14891

I am trying to do an integration test with Spring Boot, Spock-Spring and Embedded Kafka. I followed this guide, where I found "As the embedded broker is started on a random port, we can’t use the fix value in the src/main/resources/application.yml properties file. Luckily the @ClassRule sets a spring.embedded.kafka.brokers system property to the address of the embedded broker(s)." Hence I have

@ContextConfiguration
@SpringBootTest(classes = [Application], webEnvironment = SpringBootTest.WebEnvironment.RANDOM_PORT)
@DirtiesContext
@Stepwise
@ActiveProfiles("test")
class AnIntegrationTest extends Specification {

  @Autowired
  private KafkaListenerEndpointRegistry kafkaListenerEndpointRegistry

  @ClassRule
  public static KafkaEmbedded embeddedKafka = new KafkaEmbedded(1, true, "topic")

  private KafkaTemplate<String, String> template

  def setup() {
    Map<String, Object> senderProperties = KafkaTestUtils.senderProps(embeddedKafka.getBrokersAsString())
    ProducerFactory<String, String> producerFactory =
            new DefaultKafkaProducerFactory<String, String>(senderProperties)
    template = new KafkaTemplate<>(producerFactory)
    template.setDefaultTopic("topic")
    for (MessageListenerContainer messageListenerContainer : kafkaListenerEndpointRegistry
            .getListenerContainers()) {
        ContainerTestUtils.waitForAssignment(messageListenerContainer, embeddedKafka.getPartitionsPerTopic())
    }
  }

  def "test"() {
    given:
      template.sendDefault("Hello")
  }
}

In application-test.yml I have

kafka:
  bootstrapservers: ${spring.embedded.kafka.brokers}

In the application I start a Kafka consumer with spring-kafka. Stack trace is

java.lang.IllegalStateException: Failed to load ApplicationContext

    at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:124)
    at org.springframework.test.context.support.DefaultTestContext.getApplicationContext(DefaultTestContext.java:83)
    at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.injectDependencies(DependencyInjectionTestExecutionListener.java:117)
    at org.springframework.test.context.support.DependencyInjectionTestExecutionListener.prepareTestInstance(DependencyInjectionTestExecutionListener.java:83)
    at org.springframework.boot.test.autoconfigure.SpringBootDependencyInjectionTestExecutionListener.prepareTestInstance(SpringBootDependencyInjectionTestExecutionListener.java:44)
    at org.springframework.test.context.TestContextManager.prepareTestInstance(TestContextManager.java:230)
    at org.spockframework.spring.SpringTestContextManager.prepareTestInstance(SpringTestContextManager.java:50)
    at org.spockframework.spring.SpringInterceptor.interceptSetupMethod(SpringInterceptor.java:42)
    at org.spockframework.runtime.extension.AbstractMethodInterceptor.intercept(AbstractMethodInterceptor.java:28)
    at org.spockframework.runtime.extension.MethodInvocation.proceed(MethodInvocation.java:87)
    at org.spockframework.runtime.extension.MethodInvocation.proceed(MethodInvocation.java:88)
    at org.spockframework.runtime.extension.builtin.AbstractRuleInterceptor$1.evaluate(AbstractRuleInterceptor.java:37)
    at org.togglz.junit.TogglzRule$1.evaluate(TogglzRule.java:127)
    at org.spockframework.runtime.extension.builtin.TestRuleInterceptor.intercept(TestRuleInterceptor.java:38)
    at org.spockframework.runtime.extension.MethodInvocation.proceed(MethodInvocation.java:87)
    at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
    at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:68)
    at com.intellij.rt.execution.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:47)
    at com.intellij.rt.execution.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:242)
    at com.intellij.rt.execution.junit.JUnitStarter.main(JUnitStarter.java:70)
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'kafkaListenerContainerFactoryConfiguration': Injection of autowired dependencies failed; nested exception is java.lang.IllegalArgumentException: Could not resolve placeholder 'spring.embedded.kafka.brokers' in value "${spring.embedded.kafka.brokers}"
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:372)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1264)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483)
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306)
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302)
    at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:761)
    at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)
    at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:543)
    at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122)
    at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693)
    at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360)
    at org.springframework.boot.SpringApplication.run(SpringApplication.java:303)
    at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:120)
    at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:98)
    at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:116)
    ... 19 more
Caused by: java.lang.IllegalArgumentException: Could not resolve placeholder 'spring.embedded.kafka.brokers' in value "${spring.embedded.kafka.brokers}"
    at org.springframework.util.PropertyPlaceholderHelper.parseStringValue(PropertyPlaceholderHelper.java:174)
    at org.springframework.util.PropertyPlaceholderHelper.replacePlaceholders(PropertyPlaceholderHelper.java:126)
    at org.springframework.core.env.AbstractPropertyResolver.doResolvePlaceholders(AbstractPropertyResolver.java:236)
    at org.springframework.core.env.AbstractPropertyResolver.resolveRequiredPlaceholders(AbstractPropertyResolver.java:210)
    at org.springframework.core.env.AbstractPropertyResolver.resolveNestedPlaceholders(AbstractPropertyResolver.java:227)
    at org.springframework.core.env.PropertySourcesPropertyResolver.getProperty(PropertySourcesPropertyResolver.java:84)
    at org.springframework.core.env.PropertySourcesPropertyResolver.getProperty(PropertySourcesPropertyResolver.java:61)
    at org.springframework.core.env.AbstractEnvironment.getProperty(AbstractEnvironment.java:527)
    at org.springframework.context.support.PropertySourcesPlaceholderConfigurer$1.getProperty(PropertySourcesPlaceholderConfigurer.java:132)
    at org.springframework.context.support.PropertySourcesPlaceholderConfigurer$1.getProperty(PropertySourcesPlaceholderConfigurer.java:129)
    at org.springframework.core.env.PropertySourcesPropertyResolver.getProperty(PropertySourcesPropertyResolver.java:81)
    at org.springframework.core.env.PropertySourcesPropertyResolver.getPropertyAsRawString(PropertySourcesPropertyResolver.java:71)
    at org.springframework.core.env.AbstractPropertyResolver$1.resolvePlaceholder(AbstractPropertyResolver.java:239)
    at org.springframework.util.PropertyPlaceholderHelper.parseStringValue(PropertyPlaceholderHelper.java:147)
    at org.springframework.util.PropertyPlaceholderHelper.replacePlaceholders(PropertyPlaceholderHelper.java:126)
    at org.springframework.core.env.AbstractPropertyResolver.doResolvePlaceholders(AbstractPropertyResolver.java:236)
    at org.springframework.core.env.AbstractPropertyResolver.resolveRequiredPlaceholders(AbstractPropertyResolver.java:210)
    at org.springframework.context.support.PropertySourcesPlaceholderConfigurer$2.resolveStringValue(PropertySourcesPlaceholderConfigurer.java:172)
    at org.springframework.beans.factory.support.AbstractBeanFactory.resolveEmbeddedValue(AbstractBeanFactory.java:831)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1086)
    at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1066)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor$AutowiredFieldElement.inject(AutowiredAnnotationBeanPostProcessor.java:585)
    at org.springframework.beans.factory.annotation.InjectionMetadata.inject(InjectionMetadata.java:88)
    at org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor.postProcessPropertyValues(AutowiredAnnotationBeanPostProcessor.java:366)
    ... 36 more

Does anyone have a clue?

spring-kafka spring
4 Answers

Ensure that in each of your integration test you setup both @SpringBootTest and @EmbeddedKafka.

This is common mistake when you run multiple integration tests at once with @SpringBootTest annotation. Since you have registered DefaultKafkaConsumerFactory and DefaultKafkaProducerFactory in your application configurator bean and probably you also use @Value("${spring.kafka.bootstrap-servers}") somewhere there, you are obligated to setup @EmbeddedKafka(partitions = 1, topics = {"topic"}) everywhere you call spring-boot application to run on test server container. @EmbeddedKafka is responsible to call EmbeddedKafkaContextCustomizer to register all environment variables (among others spring.embedded.kafka.brokers) needed by kafka engine.

Without @EmbeddedKafka in your Integration Test spring runs your application on server but there is no mechanism to set up environments used by kafka engine.

user_54293100

I invoked embeddedKafka.before() in setup(). Then the spring.embedded.kafka.brokers got set and available.

I am not sure, but the problem seems like Spock Spring specific.

user_17186810

I am not familiar with Spock but a simple Java JUnit test works fine...

@SpringBootApplication
public class So47172973Application {

    public static void main(String[] args) {
        SpringApplication.run(So47172973Application.class, args);
    }

    @KafkaListener(topics = "foo")
    public void in(String in) {
        System.out.println(in);
    }

}

and

spring:
  kafka:
    bootstrap-servers: 
    - ${spring.embedded.kafka.brokers}
    consumer:
      group-id: embedded1
      auto-offset-reset: earliest

and

@RunWith(SpringRunner.class)
@SpringBootTest
public class So47172973ApplicationTests {

    @ClassRule
    public static KafkaEmbedded embeddedKafka = new KafkaEmbedded(1, true, "foo");

    @Autowired
    private KafkaTemplate<String, String> template;

    @Test
    public void test() throws InterruptedException {
        template.send("foo", "bar");
        Thread.sleep(10_000);
    }

}

A couple of things to try:

Use the annotation instead...

@RunWith(SpringRunner.class)
@SpringBootTest
@EmbeddedKafka(controlledShutdown = true, topics = "foo")
public class So47172973ApplicationTests {

    @Autowired
    private KafkaTemplate<String, String> template;

    @Test
    public void test() throws InterruptedException {
        template.send("foo", "bar");
        Thread.sleep(10_000);
    }

}

declare the broker as a @Bean...

@RunWith(SpringRunner.class)
@SpringBootTest(classes = {So47172973ApplicationTests.Config.class, So47172973Application.class})
public class So47172973ApplicationTests {

    @Autowired
    private KafkaTemplate<String, String> template;

    @Test
    public void test() throws InterruptedException {
        template.send("foo", "bar");
        Thread.sleep(10_000);
    }

    @Configuration
    public static class Config {

        @Bean
        public KafkaEmbedded embeddedKafka() {
            return new KafkaEmbedded(1, true, "foo");
        }

    }

}
user_12407630

Got the similar issue and @taro's answer didn't resolve my issue.

I fixed this issue by removing static from EmbeddedKafka object and replacing @ClassRule with @Rule annotation.

@Rule
public KafkaEmbedded embeddedKafka = new KafkaEmbedded(1, true, "topic")
user_14622820
Related