import statements in webpack 5 seem to wrap everything in module.exports = ""

I recently upgraded from webpack 4 to webpack 5 along with rails webpacker 6.0.0.rc5 and am finding that before, doing something like:

import template from 'template.html'

the template variable would be '<html></html>'

but now after this upgrade, it is:

'module.exports = "<html></html>"'

How can I tell webpack to import the text and not wrap it in a module.export?

UPDATE

I upgraded html-loader to 3.0.1, and now my template variable becomes the string:

'// Module
var code = "{{ <html></html> }} ";
// Exports
export default code;'