JUnit - run tests in a category only if specific profile is active

I have the following tests:

FirstUnitTest.java
SecondUnitTest.java
FirstIntegrationTest.java
SecondIntegrationTest.java

The unit tests are not marked with a category.

The two integration tests are marked with @Category(IntegrationTests.class).

I want by default to run all tests EXCEPT for the integration tests. If, however, a profile integration-tests-only is active, i want to run ONLY the integration tests.

I naively thought the following configuration would make this work:

<build>
    <plugins>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-surefire-plugin</artifactId>
            <configuration>
                <excludedGroups>com.example.IntegrationTests</excludedGroups>
            </configuration>
        </plugin>
    </plugins>
</build>
<profiles>
    <profile>
        <id>integration-tests-only</id>
        <build>
            <plugins>
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <configuration>
                        <groups>com.example.IntegrationTests</groups>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </profile>
</profiles>

But while running the tests without a profile does exactly what I want - run only the unit tests, if I activate the integration-tests-only profile no tests run at all.

Any ideas what I'm doing wrong?