How can I aggregate excludes from modules into the root excludes?

Question

We have a multi-module Gradle project. All of the modules are immediate subdirectories of the root project directory.

In the modules, we run JUnit and generate coverage with Jacoco. In the root, we aggregate the coverage. This all works, but currently we have to enter the exclusions twice:

MODULE1:

apply plugin: "jacoco"
jacoco {
    toolVersion = "0.8.5"
}

def static analysisExcludes() {
    return [
        "com/ourcompany/module1/*Config*",
        "com/ourcompany/module1/endpoint/exception/**",
        "com/ourcompany/module1/v2/**",
        "com/ourcompany/module1/v3/**",
        "src/generated/**",
        "src/*test*/**",
        "wrapper/dists/**"
    ]
}

def execData() {
    return files(fileTree(buildDir).include("jacoco/*.exec"))
}

jacocoTestReport {
    getExecutionData().setFrom(execData())
    afterEvaluate {
        classDirectories.setFrom(files(classDirectories.files.collect {
            fileTree(dir: it, exclude: analysisExcludes())
        }))
    }
}

jacocoTestCoverageVerification {
    getExecutionData().setFrom(execData());
    afterEvaluate {
        classDirectories.setFrom(files(classDirectories.files.collect {
            fileTree(dir: it, exclude: analysisExcludes())
        }))
    }
    violationRules {
        rule {
            limit {
                minimum = 0.93
            }
        }
    }
}

MODULE2:

apply plugin: "jacoco"
jacoco {
    toolVersion = "0.8.5"
}

def static analysisExcludes() {
    return [
        "com/ourcompany/module2/*IgnoreMe*"
        "src/generated/**",
        "src/*test*/**",
        "wrapper/dists/**"
    ]
}

def execData() {
    return files(fileTree(buildDir).include("jacoco/*.exec"))
}

jacocoTestReport {
    getExecutionData().setFrom(execData())
    afterEvaluate {
        classDirectories.setFrom(files(classDirectories.files.collect {
            fileTree(dir: it, exclude: analysisExcludes())
        }))
    }
}

jacocoTestCoverageVerification {
    getExecutionData().setFrom(execData());
    afterEvaluate {
        classDirectories.setFrom(files(classDirectories.files.collect {
            fileTree(dir: it, exclude: analysisExcludes())
        }))
    }
    violationRules {
        rule {
            limit {
                minimum = 0.87
            }
        }
    }
}

ROOT:

apply plugin: "jacoco"
jacoco {
    toolVersion = "0.8.5"
}

// TODO: Eliminate double-entry bookkeeping by getting the excludes from the modules.
def static aggregatedAnalysisExcludes() {
    return [
        "**/com/ourcompany/module1/*Config*",
        "**/com/ourcompany/module1/endpoint/exception/**",
        "**/com/ourcompany/module1/v2/**",
        "**/com/ourcompany/module1/v3/**",
        "**/com/ourcompany/module2/*IgnoreMe*"
        "**/src/generated/**",
        "**/src/*test*/**",
        "**/wrapper/dists/**"
    ]
}

def execData() {
    return files(fileTree(rootDir).include("**/build/jacoco/*.exec"))
}

def includedClasses() {
    return files(fileTree(dir: rootDir, include: "**/build/classes/**", exclude: aggregatedAnalysisExcludes()))
}

task jacocoAggregateReport(type: JacocoReport) {
    getExecutionData().setFrom(execData());
    afterEvaluate {
        classDirectories.setFrom(includedClasses())
    }
}

task jacocoAggregateTestCoverageVerification(type: JacocoCoverageVerification) {
    getExecutionData().setFrom(execData());
    afterEvaluate {
        classDirectories.setFrom(includedClasses())
    }
    violationRules {
        rule {
            limit {
                minimum = 0.91
            }
        }
    }
}

task mergeJacocoExecData(type: JacocoMerge) {
    setExecutionData(execData());
    setDestinationFile(new File("build/jacoco/all.exec"))
}

apply plugin: "org.sonarqube"

def junitResultsDirs() {
    def dirs = []
    rootDir.eachDirRecurse { dir ->
        if (dir.absolutePath.matches("^.*/build/test-results/(test|(component|integration)Test)\$")) {
            dirs << dir
        }
    }
    return dirs;
}

sonarqube {
    properties {
        property "sonar.projectName", "Multimodule Repo"
        property "sonar.projectKey", "multimodule-repo"
        property "sonar.java.coveragePlugin", "jacoco"
        property "sonar.jacoco.reportPath", "build/jacoco/all.exec"
        property "sonar.junit.reportsPath", junitResultsDirs()
        property "sonar.issuesReport.html.enable", true
        property "sonar.issuesReport.console.enable", true
        property 'sonar.exclusions', aggregatedAnalysisExcludes()
    }
}

We'd like to somehow iterate the modules to get their excludes, and combine them into the aggregated excludes, adding the "**/" prefixes that make the paths relative to the root.

In pseudo-code:

def aggregatedAnalysisExcludes() {
    return rootDir.modules.excludes.withPrefixes("**/")
}

or:

def aggregatedAnalysisExcludes() {
    def excludes = []
    for (Module m : rootDir.modules) {
        excludes << m.excludes.withPrefixes("**/")
    }
    return excludes
}

How do we write the code for real?

Note: Assuming there is a way to do what we want, the resulting list will contain duplicates of src/generated, src/test, and wrapper/dists. We tried creating lists with duplicates to see if that matters, and it does not. The excludes work correctly even when there are duplicates. But, if there's a clean declarative way to remove duplicates, that would be a nice addition to the solution.

user_1005481 · Answer

I understand that you want to make sure that Jacoco excludes which were made on the subproject-level propagate to the aggregation level automatically.

This can be achieved by making sure that filetrees are not evaluated too early, and of course asking the JacocoReport tasks about how they were configured. I found this Gist with the solution, and wrote a small plugin which creates an aggregated report, called gradle-jacoco-log (look inside to find the source code which explicitly performs the aggregation, including exclusions).

You basically add to your build.gradle:

plugins {
    id 'org.barfuin.gradle.jacocolog' version '1.1.0'
}

and then run

gradle jacocoAggregatedReport

Here's a full example project which has subprojects with JacocoReport exclusions that are automatically aggregated.

user_1461450 · Answer

I wanted a more Groovy-ish solution (declarative, closures), and thanks to @barfuin, was able to finally get there.

In the modules, add this after defining the excludes:

ext {
    analysisExcludes = analysisExcludes()
}

In the root, add this:

def aggregatedAnalysisExcludes() {
    evaluationDependsOnChildren()
    def excludes = new HashSet<>()
    subprojects.each {
        subproject -> subproject.property("analysisExcludes").each {
            exclude -> excludes << "**/" + exclude;
        }
    }
    return (String[])excludes.stream().sorted().toArray()
}

The (String[]) cast is annoying, but without it the types don't match when assigning to exclude: later. Changing to new HashSet() doesn't help, and .toArray(String[]::new) gets a compile error.

user_1461450 · Answer

Cleaned-up version of the excludes aggregator:

def aggregatedAnalysisExcludes() {
evaluationDependsOnChildren()
def excludes = new HashSet<>()
subprojects.each {
    subproject ->
        subproject.findProperty("analysisExcludes").each {
            exclude -> excludes << "**/" + exclude
        }
}
return excludes.stream().sorted().toArray({ length -> new String[length] })

}

How can I aggregate excludes from modules into the root excludes?

3 Answers