Private Swift and ObjC dependencies inside framework

In framework we are using Swift and Objective-C, so that Swift instances are instantiated within Objective-C methods. To be able to do so, we had to make those Swift files public/open, so they appear in SWIFT_INSTALL_OBJC_HEADER (OurModule-Swift.h), which is then imported by Objective-C. The problem is that we'd like to hide those Swift classes from public interface of framework. Apple says:

For framework targets, only declarations with the public or open modifier appear in the generated header. You can still use Swift methods and properties that are marked with the internal modifier from within the Objective-C part of your framework, as long they are declared within a class that inherits from an Objective-C class. For more information on access-level modifiers, see Access Control in The Swift Programming Language (Swift 3.1).

So in theory it should be possible to have internal Swift classes created from Objective-C, but it doesn't seem to work. Any ideas?