I made the perceptron with keras and sklearn.
it receive 917 sentences and 917 labels.
However it returns error INVALID_ARGUMENT: indices[1] = [0,447] is out of order. Many sparse ops require sorted indices
I googled around and found, in this case, I should do something like tf.sparse.reorder ,however, how?
I am reading SparceTensor but still in vague.
def train(self,texts,labels):
vectorizer = TfidfVectorizer(tokenizer=self._tokenize,ngram_range=(1,2))
tfidf = vectorizer.fit_transform(texts)
feature_dim = len(vectorizer.get_feature_names())
n_labels = max(labels) + 1
mlp = Sequential()
mlp.add(Dense(units=32,input_dim=feature_dim,activation='relu'))
mlp.add(Dense(units=n_labels,activation='softmax'))
mlp.compile(loss='categorical_crossentropy',optimizer='adam')
labels_onehot = to_categorical(labels,n_labels)
print(tfidf.shape) // (917, 2783)
print(labels_onehot.shape) // (917, 49)
mlp.fit(tfidf,labels_onehot,epochs=100)// here occurs error
The error stacktrace is following
2021-12-07 04:21:48.012640: W tensorflow/core/framework/op_kernel.cc:1745] OP_REQUIRES failed at serialize_sparse_op.cc:384 : INVALID_ARGUMENT: indices[1] = [0,447] is out of order. Many sparse ops require sorted indices.
Use `tf.sparse.reorder` to create a correctly ordered copy.
Traceback (most recent call last):
File "manage.py", line 22, in <module>
main()
File "manage.py", line 18, in main
execute_from_command_line(sys.argv)
File "/Users/whitebear/anaconda3/envs/scrapy/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
utility.execute()
File "/Users/whitebear/anaconda3/envs/scrapy/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/whitebear/anaconda3/envs/scrapy/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv
self.execute(*args, **cmd_options)
File "/Users/whitebear/anaconda3/envs/scrapy/lib/python3.8/site-packages/django/core/management/base.py", line 398, in execute
output = self.handle(*args, **options)
File "/Users/whitebear/MyCode/httproot/aitext/defapp/management/commands/main_cmd.py", line 25, in handle
dialogue_agent.train(training_data['text'],training_data['label'])
File "/Users/whitebear/MyCode/httproot/aitext/aitext/helpers/dialogue_agent.py", line 57, in train
mlp.fit(tfidf,labels_onehot,epochs=100)
File "/Users/whitebear/anaconda3/envs/scrapy/lib/python3.8/site-packages/keras/utils/traceback_utils.py", line 67, in error_handler
raise e.with_traceback(filtered_tb) from None
File "/Users/whitebear/anaconda3/envs/scrapy/lib/python3.8/site-packages/tensorflow/python/framework/ops.py", line 7107, in raise_from_not_ok_status
raise core._status_to_exception(e) from None # pylint: disable=protected-access
tensorflow.python.framework.errors_impl.InvalidArgumentError: indices[1] = [0,447] is out of order. Many sparse ops require sorted indices.
Use `tf.sparse.reorder` to create a correctly ordered copy.