How can I download NLTK corpora via `requirements.txt` using `pip install -r requirements.txt`?

Question

One can download NLTK corpora punkt and wordnet via the command line:

python3 -m nltk.downloader punkt wordnet

How can I download NLTK corpora via requirements.txt using pip install -r requirements.txt?

For example one can download spacy models requirements.txt using pip install -r requirements.txt by adding the URL of the model (e.g. https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.0.0/en_core_web_sm-2.0.0.tar.gz#egg=en_core_web_sm==2.0.0 in requirements.txt)

user_7976758 · Answer

How can I download NLTK corpora via requirements.txt

Short answer: no way.

The URL for spacy models points to a Python package (setup.py and all that) so it can be downloaded and installed by pip. There are no such pip-installable packages for NLTK data. nltk.downloader downloads data in its own format.

user_11989409 · Answer

There is no way to actually do this via a requirements.txt file. However, if it is necessary for you to use NLTK for wordnet and punkt what you can do is have 2 files. And download the nltk data in one and import that file into your main file. For example,

nltkmodules.py:

import nltk

nltk.download('wordnet')
nltk.download('punkt')

main.py:

import nltkmodules

# Rest of Code goes here

In your requirements.txt, you can just include:

nltk==3.5

How can I download NLTK corpora via `requirements.txt` using `pip install -r requirements.txt`?

2 Answers