BackHandler does not go back more than 1 screen

I have this code on each of my screens. Pressing the android back button goes back 1 screen. Pressing android back button again does not do anything. Expected result would be to keep going back as long as there are more screens in the stack. What's missing?

componentWillMount() {
    BackHandler.addEventListener('hardwareBackPress', () => {
        this.props.navigation.goBack();
        return true;
    });
}

componentWillUnmount() {
  BackHandler.removeEventListener('hardwareBackPress')
}