Error in Installing spacy en_core_web_lg on Heroku app

user_12045111 0
Viewed 1642

Am deploying my ML model on Heroku using Django, I need en_core_web_lg for my application but couldn't install it

My requirements.txt is like:

..
..
djangorestframework==3.12.2
en-core-web-lg==2.3.1
en-core-web-sm==2.3.1
gunicorn==20.0.4
heroku==0.1.4
..

The Error is:

 ERROR: Could not find a version that satisfies the requirement en-core-web-lg==2.3.1 (from -r /tmp/build_c3075f3c_/requirements.txt (line 14)) (from versions: none)
       ERROR: No matching distribution found for en-core-web-lg==2.3.1 (from -r /tmp/build_c3075f3c_/requirements.txt (line 14))
 !     Push rejected, failed to compile Python app.
 !     Push failed
spacy heroku django python
2 Answers

Your error is because there is no pip library named en-core-web-lg. Keep in mind that you cannot put en-core-web-lg in the requirements.txt since it is not a library that you can install using pip. They are models that spacy uses.

The accepted answer is fine. However, if you want a complete automatic deployment, you can also add something like this into your code:

    try:
        nlp = spacy.load("en_core_web_md")
    except: # If not present, we download
        spacy.cli.download("en_core_web_md")
        nlp = spacy.load("en_core_web_md")

This will download the models automatically if they are not present. And, it will not require you to use the Heroku terminal. Only spacy on the requirements.txt file will be necessary.

Finally, keep in mind that spacy models are loaded in memory (RAM). The lg model (about 700MB) is too big to be fitted in a Free, Hobby or Standard 1X dynos (512MB of RAM). So it will fail to load and the dyno will be killed with R15 (https://devcenter.heroku.com/articles/error-codes#r15-memory-quota-vastly-exceeded).

More info about dynos: https://www.heroku.com/dynos

user_94539040

I don't know why it is not installing direct from requirements.txt file. There is another way it is not right way but the work will go on. First you have to remove the package from requirements.txt file for which the error is coming. Then push the app on Heroku, once your app have come to Heroku then write this code either from the terminal or from the Heroku dashboard.

If you are using terminal then:

heroku run bash

Then run:

pip install spacy

and then install which requiremts you want from spacy

python -m spacy download en_core_web_lg

if you are using Heroku dashboard then : first go to your Heroku dashboard click on your app and then at top right click on More and select the Run Console example image:

enter image description here than this interface will come up enter image description here in this you have to click on bash or type bash and run after this you can put same cammand and install.

user_144578330
Related