Hugging Face model Bio_ClinicalBERT producing 404 error

user_11610317 0
Viewed 1419

I'm building a Named Entity Recognition (NER) model using the Hugging Face implementation of emilyalsentzer/Bio_ClinicalBERT. Up to today, I've had no issues with the model. I'm hopeful that someone can help me understand why it's currently not working as expected.

Question 1 - today, trying to train using:

MODEL_NAME = 'emilyalsentzer/Bio_ClinicalBERT'
model = text.sequence_tagger('bilstm-bert', preproc, bert_model=MODEL_NAME)

results in this error: 404 Client Error: Not Found for url: https://huggingface.co/emilyalsentzer/Bio_ClinicalBERT/resolve/main/tf_model.h5

Does Hugging Face offer any kind of health check to ascertain the status of their models?

Question 2 - working with files (model.h5, model.json, and preproc.sav) I'd saved from earlier training iterations, I'm getting the same 404 error shown above. I don't understand wherein these files the call to Hugging Face is occurring. It doesn't seem to be in the .json, and the .h5 and .sav file formats are hard to inspect. Read more about what these files are: https://medium.com/analytics-vidhya/how-to-deploy-your-neural-network-model-using-ktrain-ae255b134c77

Back in February, I'd used these exact model.h5, model.json, and preproc.sav files to run the NER app using Streamlit, no problem. Not sure if this is temporary issue with Bio_ClinicalBERT or if I need to retool my original approach due to potentially permanent problems with this transformer model.

huggingface-transformers nlp python
2 Answers

Judging from the URL you provided for Question 1, I understand that you are trying to load pre-trained weights for a TensorFlow model (tf_model.h5)

However, the pre-trained model you are trying to load is only available in a PyTorch version. Go on the link and look at the tags at the top of the page. This model only has a PyTorch tag, not a Tensorflow tag.

It's my understanding that in this case you should load pre-trained weights from the PyTorch checkpoint. To that end, you would add this parameter to the function that does the loading:

from_pt=True
user_3710160

As mentioned in this post in the ktrain repo, you should be able to safely ignore the error.

The 404 error simply means that transformers was not able to find a Tensorflow version of this particular model (which is being requested by ktrain). If you go to the Hugging Face model hub, you'll see that the model you're using was only uploaded as a PyTorch model.

In cases like this, the PyTorch version of the model checkpoint will be automatically downloaded by ktrain and then loaded as a Tensorflow model for training/fine-tuning. If you type model.summary(), it should show that the model was loaded successfully.

The reason you didn't see the error before is because newer versions of Hugging Face transformers show the 404 error, I think.

user_135506990
Related