I've created a Github repo that has all the source code related to this question. I'm able to load build/index.html in the browser after running npm install. However, when I view the sources tab in Chrome, I don't see the source maps associated with the .tsx files.
Here's a screenshot of what I see: 
I've added sourceMap: true in my tsconfig.json as well as debug: true and devtool: "source-map" in my webpack.config.json. Any thoughts on what I'm missing here?
EDIT:
This might be a silly question, but do you have to use webpack-dev-server in order to see the sourcemaps?