How to merge source sets while sharing dependencies to each other

I'd like to publish a library with two different API versions where both use the same core code underneath. I tried shading/shadowing but have struggles getting the visibility right (I'd like to hide the core code from the API user). So I wanted to achieve my goals by having different source sets and configurations:

sourceSets {
    // the `main` source set acts as the common code base for `api` and `api2`

    api {
        java {
            srcDir 'src/api/java'
            // Includes classes from `main`:
            compileClasspath += sourceSets.main.output
            runtimeClasspath += sourceSets.main.output
        }
    }
    api2 {
        java {
            srcDir 'src/api2/java'
            // Includes classes from `main`:
            compileClasspath += sourceSets.main.output
            runtimeClasspath += sourceSets.main.output
        }
    }
}

configurations {
    common {
        canBeResolved = true
        canBeConsumed = false
    }

    // These art the configurations used both for being consumed with `project(...)` or published:
    exposedApi {
        canBeResolved = true
        canBeConsumed = true
        extendsFrom common
    }
    exposedApi2 {
        canBeResolved = true
        canBeConsumed = true
        extendsFrom common
    }
}

task apiJar(type: Jar) {
    group = 'build'
    from configurations.exposedApi
    baseName = 'api'
}

task api2Jar(type: Jar) {
    group = 'build'
    from configurations.exposedApi2
    baseName = 'api2'
}

publishing {
    publications {
        api(MavenPublication) {
            artifact apiJar
            artifactId 'mylib-api'
        }
        api2(MavenPublication) {
            artifact api2Jar
            artifactId 'mylib-api2'
        }

    }
}

dependencies {
    common sourceSets.main.output
    exposedApi sourceSets.api.output
    exposedApi2 sourceSets.api2.output
}

If I want to use one of these APIs I can easily use project(path: ':mylib', configuration: 'exposedApi2') or use one of the published Maven artifacts and it works nicely.

But as soon as I change classes in the main source set to internal in order to achieve proper encapsulation of the main code, the API code won't compile anymore:

Cannot access 'SomeClassInMain': it is internal in '' (<-- yes, it really shows nothing in the '')

I also tried to merge the source set into one, so there is technically not really a main source set anymore:

sourceSets {
    api {
        java {
            srcDirs('src/api/java', 'src/main/java')
        }
    }
    api2 {
        java {
            srcDirs('src/api2/java', 'src/main/java')
        }
    }
}

That now works all as intended, no compilation errors, calls from the API to main work as expected and the classes in main even have internal visibility. But unfortunately IntelliJ seems to not pick up the fact that classes in main are really part of the same source set. I get an error (Unresolved reference: SomeClassInMain) in the IDE every time I mention a class from the main sources and of course no auto-completion would work, too, making the solution somehow not really practical in the end.

So just to sum up the goal:

• it's important that the main sources are accessible to the API
• but not to the user using the API (or the Maven publication) – the only thing the user should be facing is the API
• If possible, I'd like to not put the API and main code in separate modules and publish them separately for encapsulation reasons
• I tried a shading/shadowing (fat/uber JAR) approach but I haven't managed to reduce the visibility to internal in the main sources
• I'm new to the topic of these complicated kinds of build configurations so maybe I simply have chosen the wrong approach. Maybe there's a better one which I haven't yet managed to find?

Many, many thanks in advance!