Spring boot @Configuration with @ConditionalOnClass not working

I have created a module with JMSConnectionConfig as given below.

@ConditionalOnClass(JmsTemplate.class)
@Configuration
@Log4j2
public class JMSConnectionConfig {
    ...............
    ...............
    ...............
}

I have two SpringBoot apps. One application requires JMS configuration with Spring JMS dependency and the other application doesn't require JMS. For that reason, I have added @ConditionalOnClass annotation on JMSConnectionConfig to load when it finds JmsTemplate in the class path. Though the JMSConnectionConfig dependency was there in the classpath for second application, I was expecting the bean won't load as the JmsTemplate will not be there in the class path. But, I am not getting that behavior. What is the configuration I am missing here to load the bean conditionally?