running springboot inside of spark context

Question

I'm having a rather complex app, which takes a lot of different parameters and is based on springboot.

We now want to evaluate all possible combinations, using a hyper-opt approach. Which is based on apache spark + the eharmony spotz framework.

We have a simple objective:

  def buildContext(point: Option[Point], profiles: List[String]) = {

    val (newProfiles, location) = point match {
      case Some(x) =>
        val (newProfile, location) = generateNewProfile(x)
        val newProfiles = newProfile :: profiles
        (newProfiles, Option(location))
      case None =>
        (profiles, None)
    }


    val resourceLoader = new DefaultResourceLoader()
    val app = new SpringApplication(classOf[EvalutionConfiguration])
    app.setAdditionalProfiles(newProfiles: _*)
    app.setWebApplicationType(WebApplicationType.NONE)
    app.setBannerMode(Banner.Mode.OFF)
    app.setResourceLoader(resourceLoader)

    app.addListeners((event: ApplicationEnvironmentPreparedEvent) => {
      val env: ConfigurableEnvironment = event.getEnvironment
      location match {
        case Some(location) =>
          val p = new Properties()
          p.load(new FileInputStream(location))
          env.getPropertySources.addFirst(new PropertiesPropertySource("hyperopt point", p))
        case None =>
      }
    })
    val context: ConfigurableApplicationContext = app.run()
    context
  }

where we build a context, overwrite some properties (which are our configuration parameters to evaluate)

  
  final override def apply(point: Point): Double = {
      val context = buildContext(Option(point), profiles.toList)
      //get a bean
      //do some work with it
  }

now due to spark, this all happens in parallel across several cores ( and later physical machines )

and I'm randomly getting errors like the following:

Job aborted due to stage failure: Task 0 in stage 0.0 failed 1 times, most recent failure: Lost task 0.0 in stage 0.0 (TID 0) (wohlgemuth-thinkpad-t15-gen-2i.lan executor driver): org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'confirmExistenceBySimilarity': Unsatisfied dependency expressed through field 'identify'; nested exception is org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'simpleIdentify' defined in file [/home/wohlgemuth/IdeaProjects/wcmc/pipeline/core/core-api/target/classes/edu/ucdavis/fiehnlab/ms/carrot/core/api/compound/identify/SimpleIdentify.class]: Initialization of bean failed; nested exception is org.springframework.aop.framework.AopConfigException: Could not generate CGLIB subclass of class edu.ucdavis.fiehnlab.ms.carrot.core.api.compound.identify.SimpleIdentify: Common causes of this problem include using a final class or a non-visible class; nested exception is org.springframework.cglib.core.CodeGenerationException: java.lang.LinkageError-->loader 'app' attempted duplicate class definition for edu.ucdavis.fiehnlab.ms.carrot.core.api.compound.identify.SimpleIdentify$$EnhancerBySpringCGLIB$$839bef17. (edu.ucdavis.fiehnlab.ms.carrot.core.api.compound.identify.SimpleIdentify$$EnhancerBySpringCGLIB$$839bef17 is in unnamed module of loader 'app')

which is clearly somehow related to springboot being loaded in the same class loader more than once and so CGLIB enhancing the same class more than once.

Any ideas?

running springboot inside of spark context

0 Answers