Fontawesome webfonts sometimes not loading since webpack build

I just migrated the old css and js build process in our project from grunt (and css in less) to webpack 5 (and css in scss) and everything seems fine. There is one problem, however: Sometimes instead of looking like this

Here's what the heart looks like in DevTools:

.fa-heart:before {
    content: "";
}

the fontawesome webfonts don't seem to be loading correctly and instead load in like this

The heart looks like this when I inspect it in DevTools

.fa-heart:before {
    content: "";
}

I would have expected to see

.fa-heart:before {
    content: "\f004";
}

and when I change it to that manually in the DevTools, it works.

This can be reproduced when I start a lighthouse audit in my browser: The very last load when lighthouse is done will always look like this.

I have honestly no idea what to even google about this problem. I tried using webfontloader to load the font but this did not help with the problem.

I am including fontawesome like this in my main.scss:

@import "~@fortawesome/fontawesome-pro/scss/fontawesome";
@import "~@fortawesome/fontawesome-pro/scss/light";
@import "~@fortawesome/fontawesome-pro/scss/solid";

Here's a simplified (fewer entries) version of my webpack config:

module.exports = {
  entry: {
    './dist/styles.min': { import: './src/sass/main.scss' },
    './dist/scripts.min.js': { import: './src/js/scripts.js' },
  }
  module: {
    rules: [
      {
        test: require.resolve("jquery"),
        loader: "expose-loader",
        options: {
          exposes: ["$", "jQuery", "jquery"],
        },
      },
      {
        test: /\.s?[ac]ss$/i,
        use: [
          // Compiles Sass to CSS
          MiniCssExtractPlugin.loader,
          "css-loader",
          "sass-loader",
        ],
      },
      {
        test: /\.jsx?$/,
        exclude: /(node_modules|bower_components)/,
        use: {
          loader: "babel-loader",
          options: {
            presets: [["@babel/preset-env", { debug: false, useBuiltIns: "usage", corejs: 3 }], "@babel/preset-react"],
            plugins: [["@babel/plugin-transform-runtime", { corejs: 3 }]],
          },
        },
      },
      {
        test: /(ignore)/,
        use: {
          loader: "ignore-loader",
        },
      },
      {
        test: /\.(png|jpe?g|gif|svg|ttf|eot|woff2?)$/i,
        use: [
           {
            loader: "file-loader",
            options: {
              outputPath: "dist/assets/",
              publicPath: "/assets/",
            },
          },
        ],
      },
    ]
  },
  output: {
    path: __dirname,
    filename: "[name]?[contenthash]",
  },
  resolve: {
    extensions: [".jsx", ".js", ".json"],
  },
  plugins: [
    new MiniCssExtractPlugin({
      filename: "[name].css?[contenthash]",
    }),
  ],
optimization: {
    minimize: !DEV_MODE,
    minimizer: [
      "...",
      new TerserPlugin({ extractComments: false }),
      new CssMinimizerPlugin(),
    ],
    splitChunks: {
      cacheGroups: {
        polyfills: {
          test: /[\\/]node_modules[\\/](@babel|core-js|regenerator|promise)/,
          name: "./dist/polyfills.min.js",
          chunks: "all",
        },
        vendor: {
          test: /[\\/]node_modules[\\/](slick|bootstrap|jquery)/,
          name: "./dist/vendor.min.js",
          chunks: "all",
        },
      },
    },
  },
}