Maven Shading Multi-Module Aggregation and Inheritance

user_4417924 0
Viewed 1047

Context

I have an IntelliJ project with multiple modules. I am using the maven-shade-plugin to jar with dependencies whilst reducing jar size. The project structure is as follows;

  • top-level parent (pom) (aggregates api and world-teleport)
    • api (inherits parent)
    • world-teleport (inherits parent, api)

Pom(s)

Parent

   <groupId>com.jsonmack</groupId>
    <artifactId>mcplugins</artifactId>
    <packaging>pom</packaging>
    <version>1.10-SNAPSHOT</version>

    <repositories>
        <repository>
            <id>spigot-repo</id>
            <url>https://hub.spigotmc.org/nexus/content/repositories/snapshots/</url>
        </repository>
    </repositories>

    <modules>
        <module>api</module>
        <module>world-teleport</module>
    </modules>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-compiler-plugin</artifactId>
                <configuration>
                    <source>1.8</source>
                    <target>1.8</target>
                </configuration>
                <version>3.8.1</version>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>3.2.2</version>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                        <configuration>
                            <minimizeJar>true</minimizeJar>
                            <artifactSet>
                                <excludes>
                                    <exclude>junit:junit</exclude>
                                </excludes>
                            </artifactSet>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>io.github.classgraph</groupId>
                <artifactId>classgraph</artifactId>
                <version>4.8.68</version>
            </dependency>

            <dependency>
                <groupId>org.spigotmc</groupId>
                <artifactId>spigot-api</artifactId>
                <version>1.15.2-R0.1-SNAPSHOT</version>
            </dependency>

            <dependency>
                <groupId>junit</groupId>
                <artifactId>junit</artifactId>
                <version>4.13</version>
                <scope>test</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

api

    <parent>
        <artifactId>mcplugins</artifactId>
        <groupId>com.jsonmack</groupId>
        <version>1.10-SNAPSHOT</version>
    </parent>

    <modelVersion>4.0.0</modelVersion>
    <artifactId>api</artifactId>
    <packaging>jar</packaging>

    <dependencies>
        <dependency>
            <groupId>io.github.classgraph</groupId>
            <artifactId>classgraph</artifactId>
        </dependency>

        <dependency>
            <groupId>org.spigotmc</groupId>
            <artifactId>spigot-api</artifactId>
        </dependency>
    </dependencies>

world-teleport

   <parent>
        <groupId>com.jsonmack</groupId>
        <artifactId>mcplugins</artifactId>
        <version>1.10-SNAPSHOT</version>
    </parent>

    <dependencies>
        <dependency>
            <groupId>com.jsonmack</groupId>
            <artifactId>api</artifactId>
            <version>1.10-SNAPSHOT</version>
        </dependency>

        <dependency>
            <groupId>io.github.classgraph</groupId>
            <artifactId>classgraph</artifactId>
        </dependency>

        <dependency>
            <groupId>org.spigotmc</groupId>
            <artifactId>spigot-api</artifactId>
        </dependency>

        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
        </dependency>
    </dependencies>

The problem

Everytime I perform a clean and a new install or package it is indicated to me that there are duplicate dependencies.

The error/warning(s)

[INFO] 
[INFO] --------------------< com.jsonmack:world_teleport >---------------------
[INFO] Building world_teleport 1.8-SNAPSHOT                               [7/7]
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ world_teleport ---
[INFO] Deleting /Users/Business/Documents/workspace/mcplugins/world_teleport/target
[INFO] 
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ world_teleport ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 2 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:compile (default-compile) @ world_teleport ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 19 source files to /Users/Business/Documents/workspace/mcplugins/world_teleport/target/classes
[INFO] 
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @ world_teleport ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory /Users/Business/Documents/workspace/mcplugins/world_teleport/src/test/resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.8.1:testCompile (default-testCompile) @ world_teleport ---
[INFO] Changes detected - recompiling the module!
[INFO] 
[INFO] --- maven-surefire-plugin:2.12.4:test (default-test) @ world_teleport ---
[INFO] 
[INFO] --- maven-jar-plugin:3.2.0:jar (default-jar) @ world_teleport ---
[INFO] Building jar: /Users/Business/Documents/workspace/mcplugins/world_teleport/target/world_teleport-1.8-SNAPSHOT.jar
[INFO] 
[INFO] --- maven-shade-plugin:3.2.2:shade (default) @ world_teleport ---
[INFO] Including com.jsonmack:api:jar:1.10-SNAPSHOT in the shaded jar.
[INFO] Including io.github.classgraph:classgraph:jar:4.8.68 in the shaded jar.
[INFO] Including org.spigotmc:spigot-api:jar:1.15.2-R0.1-SNAPSHOT in the shaded jar.
[INFO] Including commons-lang:commons-lang:jar:2.6 in the shaded jar.
[INFO] Including com.google.guava:guava:jar:21.0 in the shaded jar.
[INFO] Including com.google.code.gson:gson:jar:2.8.0 in the shaded jar.
[INFO] Including net.md-5:bungeecord-chat:jar:1.15-SNAPSHOT in the shaded jar.
[INFO] Including org.yaml:snakeyaml:jar:1.25 in the shaded jar.
[INFO] Minimizing jar com.jsonmack:world_teleport:jar:1.8-SNAPSHOT
[WARNING] /Users/Business/Documents/workspace/mcplugins/world_teleport/target/classes (Is a directory)
[WARNING] api-1.10-SNAPSHOT.jar, snakeyaml-1.25.jar define 207 overlapping classes and resources: 
[WARNING]   - META-INF/maven/org.yaml/snakeyaml/pom.properties
[WARNING]   - META-INF/maven/org.yaml/snakeyaml/pom.xml
[WARNING]   - org.yaml.snakeyaml.DumperOptions
[WARNING]   - org.yaml.snakeyaml.DumperOptions$FlowStyle
[WARNING]   - org.yaml.snakeyaml.DumperOptions$LineBreak
[WARNING]   - org.yaml.snakeyaml.DumperOptions$NonPrintableStyle
[WARNING]   - org.yaml.snakeyaml.DumperOptions$ScalarStyle
[WARNING]   - org.yaml.snakeyaml.DumperOptions$Version
[WARNING]   - org.yaml.snakeyaml.LoaderOptions
[WARNING]   - org.yaml.snakeyaml.TypeDescription
[WARNING]   - 197 more...
[WARNING] api-1.10-SNAPSHOT.jar, guava-21.0.jar define 931 overlapping classes and resources: 
[WARNING]   - META-INF/maven/com.google.guava/guava/pom.properties
[WARNING]   - META-INF/maven/com.google.guava/guava/pom.xml
[WARNING]   - com.google.common.annotations.Beta
[WARNING]   - com.google.common.annotations.GwtCompatible
[WARNING]   - com.google.common.annotations.GwtIncompatible
[WARNING]   - com.google.common.annotations.VisibleForTesting
[WARNING]   - com.google.common.base.Absent
[WARNING]   - com.google.common.base.AbstractIterator
[WARNING]   - com.google.common.base.AbstractIterator$1
[WARNING]   - com.google.common.base.AbstractIterator$State
[WARNING]   - 921 more...
[WARNING] api-1.10-SNAPSHOT.jar, gson-2.8.0.jar define 161 overlapping classes and resources: 
[WARNING]   - META-INF/maven/com.google.code.gson/gson/pom.properties
[WARNING]   - META-INF/maven/com.google.code.gson/gson/pom.xml
[WARNING]   - com.google.gson.ExclusionStrategy
[WARNING]   - com.google.gson.FieldAttributes
[WARNING]   - com.google.gson.FieldNamingPolicy
[WARNING]   - com.google.gson.FieldNamingPolicy$1
[WARNING]   - com.google.gson.FieldNamingPolicy$2
[WARNING]   - com.google.gson.FieldNamingPolicy$3
[WARNING]   - com.google.gson.FieldNamingPolicy$4
[WARNING]   - com.google.gson.FieldNamingPolicy$5
[WARNING]   - 151 more...
[WARNING] api-1.10-SNAPSHOT.jar, bungeecord-chat-1.15-SNAPSHOT.jar define 17 overlapping classes and resources: 
[WARNING]   - META-INF/maven/net.md-5/bungeecord-chat/pom.properties
[WARNING]   - META-INF/maven/net.md-5/bungeecord-chat/pom.xml
[WARNING]   - mojang-translations/en_US.properties
[WARNING]   - mojang-translations/en_us.json
[WARNING]   - net.md_5.bungee.api.ChatColor
[WARNING]   - net.md_5.bungee.api.ChatMessageType
[WARNING]   - net.md_5.bungee.api.chat.BaseComponent
[WARNING]   - net.md_5.bungee.api.chat.ClickEvent
[WARNING]   - net.md_5.bungee.api.chat.ClickEvent$Action
[WARNING]   - net.md_5.bungee.api.chat.ComponentBuilder
[WARNING]   - 7 more...
[WARNING] api-1.10-SNAPSHOT.jar, commons-lang-2.6.jar define 63 overlapping classes and resources: 
[WARNING]   - META-INF/LICENSE.txt
[WARNING]   - META-INF/NOTICE.txt
[WARNING]   - META-INF/maven/commons-lang/commons-lang/pom.properties
[WARNING]   - META-INF/maven/commons-lang/commons-lang/pom.xml
[WARNING]   - org.apache.commons.lang.ArrayUtils
[WARNING]   - org.apache.commons.lang.BooleanUtils
[WARNING]   - org.apache.commons.lang.CharRange
[WARNING]   - org.apache.commons.lang.CharRange$1
[WARNING]   - org.apache.commons.lang.CharRange$CharacterIterator
[WARNING]   - org.apache.commons.lang.CharSet
[WARNING]   - 53 more...
[WARNING] api-1.10-SNAPSHOT.jar, bungeecord-chat-1.15-SNAPSHOT.jar, classgraph-4.8.68.jar, commons-lang-2.6.jar, gson-2.8.0.jar, guava-21.0.jar, snakeyaml-1.25.jar, spigot-api-1.15.2-R0.1-SNAPSHOT.jar, world_teleport-1.8-SNAPSHOT.jar define 1 overlapping resources: 
[WARNING]   - META-INF/MANIFEST.MF
[WARNING] api-1.10-SNAPSHOT.jar, spigot-api-1.15.2-R0.1-SNAPSHOT.jar define 710 overlapping classes and resources: 
[WARNING]   - META-INF/maven/org.spigotmc/spigot-api/pom.properties
[WARNING]   - META-INF/maven/org.spigotmc/spigot-api/pom.xml
[WARNING]   - org.bukkit.Art
[WARNING]   - org.bukkit.Axis
[WARNING]   - org.bukkit.BanEntry
[WARNING]   - org.bukkit.BanList
[WARNING]   - org.bukkit.BanList$Type
[WARNING]   - org.bukkit.BlockChangeDelegate
[WARNING]   - org.bukkit.Bukkit
[WARNING]   - org.bukkit.ChatColor
[WARNING]   - 700 more...
[WARNING] api-1.10-SNAPSHOT.jar, classgraph-4.8.68.jar define 228 overlapping classes and resources: 
[WARNING]   - LICENSE-ClassGraph.txt
[WARNING]   - META-INF.versions.9.module-info
[WARNING]   - META-INF/maven/io.github.classgraph/classgraph/pom.properties
[WARNING]   - META-INF/maven/io.github.classgraph/classgraph/pom.xml
[WARNING]   - io.github.classgraph.AnnotationClassRef
[WARNING]   - io.github.classgraph.AnnotationEnumValue
[WARNING]   - io.github.classgraph.AnnotationInfo
[WARNING]   - io.github.classgraph.AnnotationInfo$AnnotationInvocationHandler
[WARNING]   - io.github.classgraph.AnnotationInfoList
[WARNING]   - io.github.classgraph.AnnotationInfoList$AnnotationInfoFilter
[WARNING]   - 218 more...
[WARNING] maven-shade-plugin has detected that some class files are
[WARNING] present in two or more JARs. When this happens, only one
[WARNING] single version of the class is copied to the uber jar.
[WARNING] Usually this is not harmful and you can skip these warnings,
[WARNING] otherwise try to manually exclude artifacts based on
[WARNING] mvn dependency:tree -Ddetail=true and the above output.
[WARNING] See http://maven.apache.org/plugins/maven-shade-plugin/
[INFO] Minimized 6106 -> 4706 (77%)

Question

What is causing this warning? What if anything is wrong with my maven pom structure?

Duplicate Question

If this is a duplicate question please feel free to provide any resources, they would be greatly appreciated.

maven-shade-plugin maven java
4 Answers

I think the problem is that you have the io.github.classgraph and the org.spigotmc dependency added in the api and the module-1 as well.

So what happens:

  1. The api module will generate the jar artifact with io.github.classgraphand the org.spigotmc dependencies included in it.
  2. The module-1 modul which depends on the api module will import the jar file.
  3. Now the module-1 module has the io.github.classgraphand the org.spigotmc dependency from the jar, and they are declared again in the module-1 module's pom.xml file.

So the solution will be to remove the io.github.classgraph and the org.spigotmc dependency from your module-1 module pom.xml:

<parent>
    <groupId>com.jsonmack</groupId>
    <artifactId>mcplugins</artifactId>
    <version>1.10-SNAPSHOT</version>
</parent>

<dependencies>
    <dependency>
        <groupId>com.jsonmack</groupId>
        <artifactId>api</artifactId>
        <version>1.10-SNAPSHOT</version>
    </dependency>

    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
    </dependency>
</dependencies>
user_132826620

Well here I would like to appreciate your proactive was of noticing that as many developers keep on adding decencies and do not bother about such warnings.

Answer my question, as you have three different modules. Will those modules be deployed differently (independently) or together in one war or jar or package?

If your answer is yes here is nothing to worry.

If your answer is no then I see io.github dependency as redundant just get rid of it.

Delete the redundant dependencies from child module or referenced module.

Sorry if there are typos, answered in mobile but will do the job for you.

user_109153840

An other idea: Try removing io.github.classgraph and the org.spigotmc dependencies from the parent pom as well.

Parent

<groupId>com.jsonmack</groupId>
<artifactId>mcplugins</artifactId>
<packaging>pom</packaging>
<version>1.10-SNAPSHOT</version>

<repositories>
    <repository>
        <id>spigot-repo</id>
        <url>https://hub.spigotmc.org/nexus/content/repositories/snapshots/</url>
    </repository>
</repositories>

<modules>
    <module>api</module>
    <module>world-teleport</module>
</modules>

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-compiler-plugin</artifactId>
            <configuration>
                <source>1.8</source>
                <target>1.8</target>
            </configuration>
            <version>3.8.1</version>
        </plugin>

        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-shade-plugin</artifactId>
            <version>3.2.2</version>
            <executions>
                <execution>
                    <phase>package</phase>
                    <goals>
                        <goal>shade</goal>
                    </goals>
                    <configuration>
                        <minimizeJar>true</minimizeJar>
                        <artifactSet>
                            <excludes>
                                <exclude>junit:junit</exclude>
                            </excludes>
                        </artifactSet>
                    </configuration>
                </execution>
            </executions>
        </plugin>
    </plugins>
</build>

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.13</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
</dependencyManagement>

world-teleport

<parent>
    <groupId>com.jsonmack</groupId>
    <artifactId>mcplugins</artifactId>
    <version>1.10-SNAPSHOT</version>
</parent>

<dependencies>
    <dependency>
        <groupId>com.jsonmack</groupId>
        <artifactId>api</artifactId>
        <version>1.10-SNAPSHOT</version>
    </dependency>

    <dependency>
        <groupId>junit</groupId>
        <artifactId>junit</artifactId>
    </dependency>
</dependencies>
user_132826620

Thank you for the help everyone. I have found the solution which is that the API module should inherit the dependencies such as spigot and classgraph from parent but the scope of those dependencies should only be provided. This will allow the dependencies to compile but be excluded from packaging when shading. This is possible because API will not be used outside of these modules, so the API will always have the dependencies required through the other modules.

api-module.pom

    <dependencies>
        <dependency>
            <groupId>io.github.classgraph</groupId>
            <artifactId>classgraph</artifactId>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>org.spigotmc</groupId>
            <artifactId>spigot-api</artifactId>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <scope>test</scope>
        </dependency>
    </dependencies>
user_44179240
Related