How does autoreload work with jupyter?

I have a file mytest.py in the working directory, with the following content:

class MyTest(object):

  def __init__(self):
    print(1)
    pass

In jupyter I have these cells.

Cell 1:

%reload_ext autoreload
%autoreload 2
import mytest

Cell 2:

x = mytest.MyTest()

If I start with a fresh kernel then it works as expected, and it gives me 1. Now if I go into mytest.py and change the print out to 2 and rerun Cell 2 I would get:

1
[autoreload of mytest failed: Traceback (most recent call last):
  File "/Users/myusername/anaconda/lib/python3.5/site-packages/IPython/extensions/autoreload.py", line 247, in check
    superreload(m, reload, self.old_objects)
NameError: name 'reload' is not defined
]
[autoreload of mytest failed: Traceback (most recent call last):
  File "/Users/myusername/anaconda/lib/python3.5/site-packages/IPython/extensions/autoreload.py", line 247, in check
    superreload(m, reload, self.old_objects)
NameError: name 'reload' is not defined
]

If I run Cell 2 again now I'd get just 1 without any errors, but it still has not used the new code.

I can't figure out how to reload the module properly without restarting the kernel, which I can't do because I have some very time-consuming code in the real program.

I have tried this in IPython in terminal. Although instantiating the class after modifying the code also gives me errors (note that it's a different error: AttributeError: 'NoneType' object has no attribute '__dict__') but it would use the updated code if I run it again. So it seems to me that this is a problem specific to jupyter.

What should I do to fix this?

EDIT (adding version info): Anaconda 2.5.0 Python 3.5.0 IPython 4.1.2 jupyter 4.1.0 Mac OS X 10.11.2