Multi Module Scala Project Include Conf Files from other modules

user_3102968 0
Viewed 77

I have a multi module scala application that I use to do ML training. I have a core module that contains some generic configuration which I would like to add in the other modules that depends on the core module.

All my configuration files are located in the resources folder and my project structure looks like this:

core
  src
    main
     resources
       application.conf
mod1
  src
    main
     resources
       application.conf
mod2
  src
    main
     resources
       application.conf

So I would like to have in my mod1 and mod2 module's application.conf file, this as the first line:

include core/application.conf

So that I can override some of the settings from there. How do I do this? For example., here is the build.sbt sample on how I define my module mod1:

// Define Sub Modules and its settings
lazy val mod1 = (project in file(MODULE_NAME_CLEANSE)).dependsOn(core% "compile->compile;test->test", config)
  .settings(
    commonSettings,
    enablingCoverageSettings,
    dockerSettings(),
    name := MODULE_1,
    description := "Clean the incoming data for training purposes"
  )
  .enablePlugins(JavaAppPackaging, DockerPlugin)
sbt scala
1 Answers

Assuming you are using Lighbend's config library.

Once built/deployed, your 2 modules will each have in their classpath the application.conf file from core module and the one from the said module. Both will be loaded but you cannot guarantee the order.

The recommended way is to name it reference.conf file in your core module instead of application.conf because that way you are sure that reference.conf is loaded with lower priority than applicayion.conf.

See https://github.com/lightbend/config#standard-behavior :

The convenience method ConfigFactory.load() loads the following (first-listed are higher priority):

  • system properties
  • application.conf (all resources on classpath with this name)
  • application.json (all resources on classpath with this name)
  • application.properties (all resources on classpath with this name)
  • reference.conf (all resources on classpath with this name)

The idea is that libraries and frameworks should ship with a reference.conf in their jar. Applications should provide an application.conf

user_53891270
Related