Have downloaded and installed the NLTK library in synpapse notebook(using pyspark), I am importing below modules from nltk
import nltk
from collections import Counter
from nltk.tokenize import word_tokenize
from pyspark.sql.functions import lower, col
from nltk.corpus import stopwords
from nltk.sentiment.vader import SentimentIntensityAnalyzer
from nltk.stem import WordNetLemmatizer
from nltk.tokenize import word_tokenize
There is no issue in importing, now I have created below definition:
def sentiment_analyse(text1):
#text1=text1.translate(str.maketrans('','',string.punctuation))
score = SentimentIntensityAnalyzer().polarity_scores(text1)
%%sql
select sentiment_analyse('Not happy') as test
I get below issue:
Error: Job aborted due to stage failure: Task 0 in stage 0.0 failed 4 times, most recent failure: Lost task 0.3 in stage 0.0 (TID 3) (vm-d5b00299 executor 1): org.apache.spark.api.python.PythonException: Traceback (most recent call last):
File "<ipython-input-4-9905581>", line 3, in sentiment_analyse
File "/home/trusted-service-user/cluster-env/env/lib/python3.8/site-packages/nltk/sentiment/vader.py", line 339, in __init__
self.lexicon_file = nltk.data.load(lexicon_file)
File "/home/trusted-service-user/cluster-env/env/lib/python3.8/site-packages/nltk/data.py", line 750, in load
opened_resource = _open(resource_url)
File "/home/trusted-service-user/cluster-env/env/lib/python3.8/site-packages/nltk/data.py", line 875, in _open
return find(path_, path + [""]).open()
File "/home/trusted-service-user/cluster-env/env/lib/python3.8/site-packages/nltk/data.py", line 583, in find
raise LookupError(resource_not_found)
LookupError:
**********************************************************************
Resource vader_lexicon not found.
Please use the NLTK Downloader to obtain the resource:
>>> import nltk
>>> nltk.download('vader_lexicon')
For more information see: https://www.nltk.org/data.html
Attempted to load sentiment/vader_lexicon.zip/vader_lexicon/vader_lexicon.txt
I have already downloaded and installed vader_lexicon but I keep getting above error