Can we load .pkl files from an external url?

user_8284987 0
Viewed 2821

I have a pkl file of 312 MB. I want to store it to an external server (S3) or a file storing service (for example, Google Drive, Dropbox or any other). When I run my model, the pkl file should be loaded from that external url. I have checked out this post but was unable to make it work.

Code: 

import urllib
import pickle

Nu_SVC_classifier = pickle.load(urllib.request.urlopen("https://drive.google.com/open?id=1M7Dt7CpEOtjWdHv_wLNZdkHw5Fxn83vW","rb"))

Error:

TypeError: POST data should be bytes, an iterable of bytes, or a file object. It cannot be of type str.

pickle python
2 Answers

The second argument of urllib.request.urlopen is the post data, not file mode, which is not needed.

import urllib.request
import pickle

Nu_SVC_classifier = pickle.load(urllib.request.urlopen("https://drive.google.com/open?id=1M7Dt7CpEOtjWdHv_wLNZdkHw5Fxn83vW"))
user_35558450

Try joblib instead of pickle, It works for me.

from urllib.request import urlopen
from sklearn.externals import joblib
Nu_SVC_classifier = joblib.load(urlopen("https://drive.google.com/open?id=1M7Dt7CpEOtjWdHv_wLNZdkHw5Fxn83vW"))
user_112444960
Related