Error loading certain NLTK modules in Google Colab

user_16682410 0
Viewed 780

I'm trying to load the comtrans module from NLTK into a Google Colab notebook, but it's giving me the following error:

[nltk_data] Downloading package comtrans to /root/nltk_data...
[nltk_data]   Package comtrans is already up-to-date!

---------------------------------------------------------------------------

LookupError                               Traceback (most recent call last)

/usr/local/lib/python3.7/dist-packages/nltk/corpus/util.py in __load(self)
     79             except LookupError as e:
---> 80                 try: root = nltk.data.find('{}/{}'.format(self.subdir, zip_name))
     81                 except LookupError: raise e

5 frames

LookupError: 
**********************************************************************
  Resource comtrans not found.
  Please use the NLTK Downloader to obtain the resource:

  >>> import nltk
  >>> nltk.download('comtrans')
  
  Searched in:
    - '/root/nltk_data'
    - '/usr/share/nltk_data'
    - '/usr/local/share/nltk_data'
    - '/usr/lib/nltk_data'
    - '/usr/local/lib/nltk_data'
    - '/usr/nltk_data'
    - '/usr/lib/nltk_data'
**********************************************************************


During handling of the above exception, another exception occurred:

LookupError                               Traceback (most recent call last)

/usr/local/lib/python3.7/dist-packages/nltk/data.py in find(resource_name, paths)
    671     sep = '*' * 70
    672     resource_not_found = '\n%s\n%s\n%s\n' % (sep, msg, sep)
--> 673     raise LookupError(resource_not_found)
    674 
    675 

LookupError: 
**********************************************************************
  Resource comtrans not found.
  Please use the NLTK Downloader to obtain the resource:

  >>> import nltk
  >>> nltk.download('comtrans')
  
  Searched in:
    - '/root/nltk_data'
    - '/usr/share/nltk_data'
    - '/usr/local/share/nltk_data'
    - '/usr/lib/nltk_data'
    - '/usr/local/lib/nltk_data'
    - '/usr/nltk_data'
    - '/usr/lib/nltk_data'
**********************************************************************

This is the code I used:

import nltk
nltk.download('comtrans')

data = nltk.corpus.comtrans.aligned_sents('alignment-en-fr.txt')
print(data[0])
print(len(data))

In other questions I saw, most people were mentioning having problems with stopwords. But in my case, stopwords are working as expected.

import nltk
nltk.download('stopwords')

words = nltk.corpus.stopwords.words('english')
print(words[10])
print(len(words))

''' output:
[nltk_data] Downloading package stopwords to /root/nltk_data...
[nltk_data]   Package stopwords is already up-to-date!
you've
179
'''

When running on my local machine both stopwords and comtrans work fine. It's just colab where comtrans isn't able to load the data. These are the values I would expect to see from the comtrans print statements:

<AlignedSent: 'Resumption of the se...' -> 'Reprise de la sessio...'>
33334

Is there some other way to try loading this data through NLTK, or am I stuck doing something like uploading the file itself from my machine and loading it through other means. If I will need a direct file upload what code will parse it from the text file into the AlignedSent object that NLTK is returning?

google-colaboratory nltk python
1 Answers

It seems Colab is downloading the package correctly, just like it claims to be. But the NLTK modules are all downloaded as zip files, that's the case for both stopwords and comtrans. In the case of stopwords it unzips it after downloading, while that unzip step is skipped for comtrans. The difference here is that locally NLTK is willing to grab the comtrans data directly from the zip file but in Colab it isn't. So because that data is only available in zip form, it rejects the operation with a "Resource not found".

All the NLTK zips I checked contain a single folder at the root level, with all the specific files for the module contained within. That folder needs to be extracted into the same location as the zip file.

In this case unzipping just needs to be done manually.

import nltk
nltk.download('comtrans')
# Data is downloaded to /root/nltk_data/corpora/comtrans.zip

from zipfile import ZipFile
file_loc = '/root/nltk_data/corpora/comtrans.zip'
with ZipFile(file_loc, 'r') as z:
  z.extractall('/root/nltk_data/corpora/')

data = nltk.corpus.comtrans.aligned_sents('alignment-en-fr.txt')
print(data[0]) # <AlignedSent: 'Resumption of the se...' -> 'Reprise de la sessio...'>
user_166824100
Related