Creating NER model with Keras and Python

user_9749124 0
Viewed 364

I have made Keras model that detects if string value is Address, Company or Date. I have used only different company names, different date formants, and different street addresses for training. So each row in my dataset have between 1 and 5 words (some words can be numbers).

For preprocessing I have used vectorizers:

transformerVectoriser = ColumnTransformer(transformers=[('vector char', CountVectorizer(analyzer='char', ngram_range=(3, 6), max_features = 2000), 'text'),
                                                        ('vector word', CountVectorizer(analyzer='word', ngram_range=(1, 1), max_features = 4000), 'text')],
                                          remainder='passthrough') # Default is to drop untransformed columns


features = transformerVectoriser.fit_transform(features)

This is my model:

model = Sequential()
model.add(Dense(100, input_dim = features.shape[1], activation = 'relu')) # input layer requires input_dim param
model.add(Dense(200, activation = 'relu'))
model.add(Dense(100, activation = 'relu'))
model.add(Dense(50, activation = 'relu'))
model.add(Dropout(0.5))
model.add(Dense(3, activation='softmax'))

I have achieved accuracy of 93%. Is it possible to use that model for detecting where is that string (Address, Company or Date) in bigger text? I think that that kind of models is called NER models (named entity recognition).

My model takes string input and decides if its a company, person or address. String input is 1-5 words long.

For example if I have text:

text = "What do you think about Amazon and their customer policy?"

How can I detect start index and end index of "Amazon", or how can I extract only Amazon from this text? So required result should be:

start_index = 21
end_index = 27

Or:

result = {'company':'Amazon'}

Or for example if the input is:

"My name is Don John and I work in Amazon Inc."

as a result i want: {"company": "Amazon Inc", "name": "Don John"}

I want to use my model to extract specific entities from text, in this case company, address or date. Basically I need some kind of location search thats based on Keras model, or searching for the pattern in large text.

keras nlp python
1 Answers

You can go two ways:

  1. Use your current model. Split your long text into 3-5 words chunks (may be with some overlap of 1-2 words), then run your classifier on each chunk. As you move your window over the text, you will get an indication if the current chunk includes the entity you are searching for. This way you will be able to reconstruct the index of the word that triggered the entity classifier.
  2. train your own NER. Then for each (sub)word you will get a classification. It is relatively easy. Use Huggingface's transformers library. You can base your code on this post (they train Turkish NER). If you don't have access to GPU, you can use Google Colab for free. If running Transformer model is computationally too expensive for you, you can try to go with CRF model. It will provide a bit lower performance, but should be faster. Refer to this example.

To train your own NER with BERT, you need a labeled dataset. Usually it is a set of sentences in which every word is labeled (e.g. ORG, PERSON, etc.) Then you tokenize your sentence - convert text into (sub)words. The tokenizer is also provided within Huggingface library.

Then you fine-tune BERT on your dataset. The process is called fine tuning because BERT model is pretrained on vast amounts of data, and you only tune it for your task. Otherwise it is a regular supervised training process: you show a sequence of tokens and their labels to the model and try to minimize overall error. As a result each token will get some label.

In context of BERT models family NER is considered a token classification task (as opposed to text sequence classification and range detection). Usually BERT based NER perform very well. Here another post on the process.

user_85058170
Related