Using babel-plugin-module-resolver with jest

I've got babel-plugin-module-resolver set up on my project and it works fine, until I run my tests.

I've set up ~/ to point to ./src.

I've tried adding the following to package.json but it still does not work

"jest": {
  "preset": "react-native",
  "moduleNameMapper": {
    "~": "./src"
  }
}