How to add OpenAPI client as a subproject?

You cannot configure it alike this, because build most certainly is an output directory, which would create a circular reference. Better try to add a new module and add that generator plugin into that module. If you can configure another module as outputDir, this could be referenced.

Even if the plugin resides in the root project, the destination needs to be a module.

The point is, that the root project always executes, opposite to module configutions.

You are trying to make build/ a project when that directory specifically is not meant to be a project directory. It's Gradle default build directory and likely 99% of other plugins and other Gradle plugins.

Simply change output directory to something else other than build/:

openApiGenerate {
    generatorName.set("java")
    inputSpec.set("$rootDir/specs/petstore.json")
    outputDir.set("$rootDir/openapi-java-client")
    apiPackage.set("org.openapi.example.api")
    invokerPackage.set("org.openapi.example.invoker")
    modelPackage.set("org.openapi.example.model")
}

Then include the project in your build with the correct syntax:

// settings.gradle
include("openapi-java-client")

However, using the org.openapi.generator seems to generate an invalid build.gradle since I get the following error:

FAILURE: Build failed with an exception.

* Where:
Build file 'C:\Users\fmate\code\example\openapi-java-client\build.gradle' line: 23

* What went wrong:
Could not compile build file 'C:\Users\fmate\code\example\openapi-java-client\build.gradle'.
> startup failed:
  build file 'C:\Users\fmate\code\example\openapi-java-client\build.gradle': 23: unexpected char: '\' @ line 23, column 35.
         main.java.srcDirs = ['src/main\java']

This obviously won't work how you wanted it to since it appears to be an issue with the Gradle plugin itself. If you just need to include the generate code in your project, then just include the generated Java code as part of your main Java source:

openApiGenerate {
    generatorName.set("java")
    inputSpec.set("$rootDir/specs/petstore.json")
    outputDir.set("$buildDir/openapi-java-client")
    apiPackage.set("org.openapi.example.api")
    invokerPackage.set("org.openapi.example.invoker")
    modelPackage.set("org.openapi.example.model")
}

tasks {
    compileJava {
        dependsOn(openApiGenerate)
    }
}

sourceSets {
    main {
        java {
            srcDir(files("${openApiGenerate.outputDir.get()}/src/main"))
        }
    }
}

But with this approach, you'll run into missing imports/dependencies. It doesn't appear this plugin offers the ability to just generate the models/POJOs only, so updating the library property to native and including some missing dependencies manually, it all works:

plugins {
    java
    id("org.openapi.generator") version "5.0.0-beta"
}

repositories {
    mavenCentral()
}

group = "io.mateo.test"

dependencies {
    implementation(platform("com.fasterxml.jackson:jackson-bom:2.11.1"))
    implementation("com.fasterxml.jackson.core:jackson-databind")
    implementation("com.fasterxml.jackson.datatype:jackson-datatype-jsr310")
    implementation("org.openapitools:jackson-databind-nullable:0.2.1")
    implementation("com.google.code.findbugs:jsr305:3.0.2")
    implementation("io.swagger:swagger-core:1.6.2")
}

openApiGenerate {
    generatorName.set("java")
    inputSpec.set("$rootDir/specs/petstore.json")
    outputDir.set("$buildDir/openapi-java-client")
    apiPackage.set("org.openapi.example.api")
    invokerPackage.set("org.openapi.example.invoker")
    modelPackage.set("org.openapi.example.model")
    library.set("native")
    configOptions.put("dateLibrary", "java8")
}

tasks {
    compileJava {
        dependsOn(openApiGenerate)
    }
}

sourceSets {
    main {
        java {
            srcDir(files("${openApiGenerate.outputDir.get()}/src/main"))
        }
    }
}

I’ve just answered a very similar question. While my answer there is not perfect, I would personally still prefer the approach suggested there – and kind of repeated here:

Suggested Approach

I would keep the builds of the modules that depend on the generated API completely separate from the build that generates the API. The only connection between such builds should be a dependency declaration. That means, you’ll have to manually make sure to build the API generating project first and only build the dependent projects afterwards.

By default, this would mean to also publish the API module before the dependent projects can be built. An alternative to this default would be Gradle composite builds – for example, to allow you to test a newly generated API locally first before publishing it. However, before creating/running the composite build, you would have to manually run the API generating build each time that the OpenAPI document changes.

Example

Let’s say you have project A depending on the generated API. Its Gradle build would contain something like this:

dependencies {
    implementation 'com.example:api:1.0'
}

Of course, the simple-java-app build described in the question would have to be adapted to produce a module with these coordinates:

openApiGenerate {
    // …
    groupId = "com.example"
    id = "api"
    version = "1.0"
}

Before running A’s build, you’d first have to run

1. ./gradlew openApiGenerate from your simple-java-app project.
2. ./gradlew publish from the simple-java-app/build/openapi-java-client/ directory.

Then A’s build could fetch the published dependency from the publishing repository.

Alternatively, you could drop step 2 locally and run A’s build with an additional Gradle CLI option:

./gradlew --include-build $path_to/simple-java-app/build/openapi-java-client/ …