Angular 4 ng build not loading raw loader

I did an ng eject on my angular 4 app and added the following to the resulting webpack.config.js (Needed it for i18n)

{
    "test": /\.xlf$/,
    "loader": 'raw-loader'     
}

But when i do ng build --base-href --prod, it complains about not having the appropriate loader for .xlf files.

WARNING in ./src/app/locale/messages.es.xlf Module parse failed: /src/app/locale/messages.es.xlf Unexpected token (1:0) You may need an appropriate loader to handle this file type.

It seems that ng build is not using the ejected webpack.config.js