Run maven plugin on all modules in parallel without dependency order

user_3252477 0
Viewed 54

I am looking for a way to run a maven plugin, like the formatter plugin or checkstyle, on all sub-modules in parallel. It would look like mvn -T 1C formatter:format with an additional magic option.
This is a bit different from running standard maven actions because each module are independent from each other regardless formatting.

For example, let's consider a project with two sub-modules lib and app. app depends on lib, implementing its underlying logic.
When compiling code, it is not possible to compile app without first compiling lib. However, when formatting code, we can safely do both in parallel.

I suspect that this is not possible because this does not follow maven architecture, oriented towards predefined life-cycles on which we hook some plugins. But I may have missed something

maven java
1 Answers

Yes you are right about the assumption to speed up things via -T1C if the plugin it does it correctly. As you can see that the formatter-maven-plugin does it correctly. As you can see the results which shows it improves the speed.

$ hyperfine -L threads 2,4,6,8,1C -p 'git co -- .' 'mvn net.revelc.code.formatter:formatter-maven-plugin:2.18.0:format -T {threads}'
Benchmark 1: mvn net.revelc.code.formatter:formatter-maven-plugin:2.18.0:format -T 2
  Time (mean ± σ):     10.111 s ±  0.623 s    [User: 17.796 s, System: 0.918 s]
  Range (min … max):    9.349 s … 11.446 s    10 runs
 
Benchmark 2: mvn net.revelc.code.formatter:formatter-maven-plugin:2.18.0:format -T 4
  Time (mean ± σ):      8.693 s ±  0.629 s    [User: 17.247 s, System: 0.862 s]
  Range (min … max):    7.827 s …  9.628 s    10 runs
 
Benchmark 3: mvn net.revelc.code.formatter:formatter-maven-plugin:2.18.0:format -T 6
  Time (mean ± σ):      8.829 s ±  0.567 s    [User: 18.438 s, System: 0.965 s]
  Range (min … max):    8.098 s …  9.803 s    10 runs
 
Benchmark 4: mvn net.revelc.code.formatter:formatter-maven-plugin:2.18.0:format -T 8
  Time (mean ± σ):      9.257 s ±  0.956 s    [User: 18.195 s, System: 0.996 s]
  Range (min … max):    8.053 s … 10.938 s    10 runs
 
Benchmark 5: mvn net.revelc.code.formatter:formatter-maven-plugin:2.18.0:format -T 1C
  Time (mean ± σ):      8.568 s ±  0.515 s    [User: 17.955 s, System: 0.942 s]
  Range (min … max):    7.682 s …  9.537 s    10 runs
 
Summary
  'mvn net.revelc.code.formatter:formatter-maven-plugin:2.18.0:format -T 1C' ran
    1.01 ± 0.10 times faster than 'mvn net.revelc.code.formatter:formatter-maven-plugin:2.18.0:format -T 4'
    1.03 ± 0.09 times faster than 'mvn net.revelc.code.formatter:formatter-maven-plugin:2.18.0:format -T 6'
    1.08 ± 0.13 times faster than 'mvn net.revelc.code.formatter:formatter-maven-plugin:2.18.0:format -T 8'
    1.18 ± 0.10 times faster than 'mvn net.revelc.code.formatter:formatter-maven-plugin:2.18.0:format -T 2'

Just as a comparison for example with versions-maven-plugin:

$ hyperfine -L threads 2,4,6,8,1C -p 'git co -- .' 'mvn versions:set -DnewVersion=1.0.0-SNAPSHOT -DgenerateBackupPoms=false -T {threads}'
Benchmark 1: mvn versions:set -DnewVersion=1.0.0-SNAPSHOT -DgenerateBackupPoms=false -T 2
  Time (mean ± σ):      1.581 s ±  0.104 s    [User: 4.641 s, System: 0.198 s]
  Range (min … max):    1.513 s …  1.863 s    10 runs
 
Benchmark 2: mvn versions:set -DnewVersion=1.0.0-SNAPSHOT -DgenerateBackupPoms=false -T 4
  Time (mean ± σ):      1.575 s ±  0.031 s    [User: 4.627 s, System: 0.190 s]
  Range (min … max):    1.527 s …  1.623 s    10 runs
 
Benchmark 3: mvn versions:set -DnewVersion=1.0.0-SNAPSHOT -DgenerateBackupPoms=false -T 6
  Time (mean ± σ):      1.572 s ±  0.034 s    [User: 4.608 s, System: 0.195 s]
  Range (min … max):    1.530 s …  1.623 s    10 runs
 
Benchmark 4: mvn versions:set -DnewVersion=1.0.0-SNAPSHOT -DgenerateBackupPoms=false -T 8
  Time (mean ± σ):      1.558 s ±  0.031 s    [User: 4.589 s, System: 0.192 s]
  Range (min … max):    1.481 s …  1.586 s    10 runs
 
Benchmark 5: mvn versions:set -DnewVersion=1.0.0-SNAPSHOT -DgenerateBackupPoms=false -T 1C
  Time (mean ± σ):      1.565 s ±  0.024 s    [User: 4.597 s, System: 0.196 s]
  Range (min … max):    1.533 s …  1.600 s    10 runs
 
Summary
  'mvn versions:set -DnewVersion=1.0.0-SNAPSHOT -DgenerateBackupPoms=false -T 8' ran
    1.00 ± 0.03 times faster than 'mvn versions:set -DnewVersion=1.0.0-SNAPSHOT -DgenerateBackupPoms=false -T 1C'
    1.01 ± 0.03 times faster than 'mvn versions:set -DnewVersion=1.0.0-SNAPSHOT -DgenerateBackupPoms=false -T 6'
    1.01 ± 0.03 times faster than 'mvn versions:set -DnewVersion=1.0.0-SNAPSHOT -DgenerateBackupPoms=false -T 4'
    1.01 ± 0.07 times faster than 'mvn versions:set -DnewVersion=1.0.0-SNAPSHOT -DgenerateBackupPoms=false -T 2'

Here you can see there is no difference by different -T ..

I want something that runs from the terminal, to perform checks in CI, to clean the codebase after git merges,

You can call some of the goals on command line via goal only. (Always the plugins must do it correctly.).

to clean the codebase after git merges,

If you have two states of code which are correctly formatted a merge will not change the formatting (assumed automatic merge will work). Except there are conflicts which could not being done automatically (This is job for IDE).

You could break the build if the formatting is not correct.

I assumed that it triggered a parallel processing of modules.

The parallelizations is based on levels of modules

That maven would follow the order of dependencies seemed logical.

Yes the order of the reactor is analyzed by Maven.

It is interesting that you mention this can be controlled per plugin. Do you have any link to that I can read, please?

A plugin can simply read the pom files on its which circumvents the information of reactor... some plugins do that (versions-maven-plugin) which means the plugin can do everything on it's own, but repeats the logic which is already implemented in Maven itself.

Update: The versions-maven-plugin does not override the reactor it does not take into account that this information exists. It's just working on it's own

user_2963280
Related