Download speed for pip install inside docker container very slow

While creating a docker image from a dockerfile or being inside the created container, pip install <package> in a conda environment is super slow.

The download speed hugely differs if I do the same without docker. I tried to change the mirror pip install -i http:pypi.doubanio.com/simple <package> and can gain slight improvements, but still different inside and outside of docker. Testing the download speed inside the container via curl -s https://raw.githubusercontent.com/sivel/speedtest-cli/master/speedtest.py | python - gives me the expected good results.