Package the model weights of a trained neural network to make it usable for transfer learning

user_10623444 0
Viewed 256

I have trained and evaluated a neural network on a movie's dataset. Basically, I am trying to predict the multi-class dependent variable movie_genre using text inputs like the plot, actors, reviews etc.

The model weights are saved in .h5 format in a local path of my computer. Let's say here C:/Users/Desktop/classification_attention_layer_model.h5.

The model structure is also saved in a separate .json file format in the same path. Let's say here classification_attention_layer_model.json.

What I want is to give the neural network weights to a colleague and use those weights for transfer learning. So for example, create a different neural network with the pre-trained word embeddings that I sent him and use those pre-trained embeddings on top of dense/hidden layers to train his own neural network on movies.

How can I achieve this? I know that one way is to just sent him the .h5 file with the trained embedding weights. But is this a professional way to do it?

I firmly believe that docker can help on this but I don't know how. Or maybe a python package may do the trick.

The only related question I found online is this one How to create a pre-trained weight model similar to Imagenet or Noisy-student?. However, they only explain how to save the model weights which I have already done.

Basically, I want to achieve the methodology of spaCy package when it uses a text to create its word vectors. This task is demonstrated on Kaggle's course NLP exercise 3:

import spacy
import numpy as np

nlp = spacy.load('en_core_web_lg')
review_data = pd.read_csv('../input/nlp-course/yelp_ratings.csv')

vectors = np.array([nlp(review_data.text).vector for idx, review in review_data.iterrows()])
    
vectors.shape #(100, 300) where 100 rows and 300 are the vectors per row
keras tensorflow neural-network python
1 Answers

If I were you I would create a small simple package and upload it to PyPI.

You could add your model as part of the package (if it isn’t large) using MANIFEST.in as described here.

On the downside, it will take disk space of users, no matter whether they’re using your software at this moment or not.

Alternatively (IMO better), upload you weights somewhere and make them downloadable through code, something like:

import yourpackage

backbone = yourpackage.yourmodel.backbone(
    download=True, 
    pretrained=True
)

This is how torchvision approaches this problem and it seems to be good solution.

Docker

I see no point in this case, unless your model depends on non pip installable software, which is unlikely.

user_108864200
Related