Webpack not able to import React component with `<svg`> tag

I have a standard React funtional component which exports a SVG as so;

SomeSVG.js

import React from 'react';

const SomeSVG = props => (
  <svg width={24} height={24} {...props}>...</svg>
)

export default SomeSVG;

When using this in Create React App, works fine. Nice inlined SVG where it was expected to be.

Now I'm trying to use this in another project, with a custom webpack setup. But I get the error;

    Module parse failed: Unexpected token (4:2)
    You may need an appropriate loader to handle this file type.
    | 
    | const SomeSVG = props => (
    >   <svg width={24} height={24} {...props}>

I imagine I need to have a webpack loader than can support these, my existing webpack settings for JS/JSX/SVG

        {
          test: /\.jsx?$/,
          exclude: /node_modules/,
          use: [
            'babel-loader?cacheDirectory=true',
            'eslint-loader'
          ]
        },

        {
          test: /\.svg$/,
          loader: 'file-loader',
          options: {
            name: 'assets_seller/[name].[ext]'
          }
        },

(I don't think the SVG part is revelant but I'll include it for completeness)

I've tried react-svg-loader as a loader as well with and without options;

        {
          test: /\.jsx?$/,
          exclude: /node_modules/,
          use: [
            'babel-loader?cacheDirectory=true',
            {
              loader: "react-svg-loader",
              options: {
                jsx: true // true outputs JSX tags
              }
            },
            'eslint-loader'
          ]
        },

But that errors as well

NonErrorEmittedError: (Emitted value instead of an instance of Error) Error in parsing SVG: Non-whitespace before first tag.

My .babelrc config

{
  "presets": [
    [
      "@babel/preset-env", {
        "modules": false
      }
    ],
    "@babel/preset-react"
  ],
  "plugins": [
    "@babel/plugin-transform-modules-commonjs",
    "transform-async-to-generator",
    "syntax-dynamic-import",
    "universal-import",
    "@babel/plugin-transform-exponentiation-operator"
  ]
}

Any ideas would be greatly appriciated.

EDIT: I upgraded Webpack and it's plugins to the latest versions beforehand.