Why pipenv is not using its wheel cache?

Whenever I install a new package (without using the --skip-lock option), pipenv downloads and (in case of non-binary dependencies) compiles all the packages from the scratch, even though the wheels are already cached in ~/.cache/pipenv. This makes the whole development process slow, since I have a lot of packages that need to be compiled from the source. Currently I download and compile my packages using pip, use pypi-server to run a local package server, and point my pipenv to it (using [[source]]). But I'm wondering if there is a better way.