I am trying to train a model to classify multi-label data set by referring this article. I am entirely new to this field and I am getting this error "AttributeError: 'numpy.ndarray' object has no attribute 'lower'"
Here is my code
reviews = pd.read_csv("/content/drive/My Drive/Interim Project Data/score.csv")
Review_labels = reviews[["natral score", "man-made score", "sport score", "special event score"]]
Review_labels.head()
def preprocess_text(sen):
# Remove punctuations and numbers
sentence = re.sub('[^a-zA-Z]', ' ', sen)
sentence = re.sub(r"\s+[a-zA-Z]\s+", ' ', sentence)
sentence = re.sub(r'\s+', ' ', sentence)
return sentence
X = []
sentences = list(reviews["User Review"])
for sen in sentences:
X.append(preprocess_text(sen))
y = Review_labels.values
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.1, random_state=123)
print(len(X_train))
print(len(X_test))
from numpy import array
from keras.preprocessing.sequence import pad_sequences
from keras.preprocessing.text import Tokenizer
tokenizer = Tokenizer(num_words=5000)
tokenizer.fit_on_texts(X)
X_train = tokenizer.texts_to_sequences(X_train)
X_test = tokenizer.texts_to_sequences(X_test)
The error is here
---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-128-ff374d8c5eb4> in <module>()
7 tokenizer.fit_on_texts(X)
8
----> 9 X_train = tokenizer.texts_to_sequences(X_train)
10 X_test = tokenizer.texts_to_sequences(X_test)
11
2 frames
/usr/local/lib/python3.6/dist-packages/keras_preprocessing/text.py in text_to_word_sequence(text, filters, lower, split)
41 """
42 if lower:
---> 43 text = text.lower()
44
45 if sys.version_info < (3,):
AttributeError: 'numpy.ndarray' object has no attribute 'lower'
here is the data set that I used in the code
It will be really helpful if someone can help with this issue.