How to unmock all modules in Jest?

Although I'm calling jest.resetModules before each test, modules mocked in one test are affecting subsequent tests. So I'm having to explicit call jest.unmock('module') in the second test. Shouldn't jest.resetModules do that already? How can I unmock everything without worrying about what has been mocked before?