Bokeh 'utf8' codec can't decode byte 0xe9 : unexpected end of data

user_8763890 0
Viewed 18270

Im using Bokeh to plot a pandas Dataframe. Following is the code:

map_options = GMapOptions(lat=19.075984, lng=72.877656, map_type="roadmap", zoom=11)
plot = GMapPlot(x_range=DataRange1d(), y_range=DataRange1d(), map_options=map_options)


plot.api_key = "xxxxx"
source = ColumnDataSource(
    data=dict(
        lat=[float(i) for i in data.lat],
        lon=[float(i) for i in data.lon],
        size=[int(i)/1000 for i in data['count']],
        ID = [i for i in data.merchant_id],
        Merchant = [str(i) for i in data.merchant_name],
        count = [float(i) for i in data['count']]
    )
)
hover = HoverTool(tooltips=[
    ("(x,y)", "($lat, $lon)"),
    ("ID", "$ID"),
    ("Name", "@Merchant"),
    ("count","$count")
])


# hover.renderers.append(circle_glyph)
plot.tools.append(hover)
circle = Circle(x="lon", y="lat", size='size', fill_color="blue", fill_alpha=0.8, line_color=None)
plot.add_glyph(source, circle)

# plot.add_layout(labels)
plot.add_tools(PanTool(), WheelZoomTool(), BoxSelectTool())
output_file("gmap_plot.html")
show(plot)

In the Hovertool using the "Name" field throws the following error:

UnicodeDecodeError: 'utf8' codec can't decode byte 0xe9 in position 6: unexpected end of data

Also commenting the "Name" field still gives me the error but there is an output plot.

Following is the dataframe I'm using:

    lat lon merchant_id count   merchant_name
0   18.539971   73.893963   757 777 Portobello
1   18.565766   73.910980   745 10193   The Wok Box
2   18.815427   76.775143   1058    2354    Burrito Factory
3   18.914633   72.817916   87  1985    Flamboyante
4   18.915794   72.824370   94  1116    Butterfly Pond
5   18.916473   72.826868   145 1010    Leo's Boulangerie
6   18.918923   72.828325   115 517 Brijwasi Sweets
7   18.928063   72.832888   973 613 Pandora's Box
8   18.928562   72.832353   101 64  La Folie Patisserie
9   18.929516   72.831860   961 6673    Burma Burma

From my knowledge, the merchant name has characters that's causing the error, but i've tried encoding the column with 'utf-8', 'ascii', etc. But I get the following error:

data['merchant_name'] = data['merchant_name'].str.encode('utf-8')

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe9 in position 6: ordinal not in range(128)

Any Idea on how to proceed ?

pandas bokeh encoding python
2 Answers

The byte 0xe9 is not in pure ascii, because it is 233 (in decadical system) and ascii has only 127 symbols. In UTF-8 it is a special byte, which introduces a charecter taking next two bytes. Thus the string is probably in another encoding. For example in latin1 and latin2 the byte 0xe9 symbolizes the letter é.

And remember, first you must decode the string. You tried encode the type str, (normal string) which does not make sense. Therefore Python tried his default decode('ascii') and you got the UnicodeDecodeError on encode method.

I didn't manage to replicate the error and also I don't see any special characters in the data you provided (especially I don't see the 0xe9 byte). So I can only guess. I would try something like this:

data['merchant_name'] = data['merchant_name'].str.decode('latin1').encode('utf-8')

And last but not least please please please, when you post your code, post the complete code with all imports and everything. I never used Bokeh, and now, when I tried to replicate your error, it was time consuming to reconstruct them. (But anyway -- at the end I managed to import everything, but I didn't get your error.)

user_30902060

I solved the problem like this

tsne_df['words'] = list(w2v_model.wv.vocab.keys())[:5000]

tsne_df['words'] = tsne_df['words'].str.decode('latin1').str.encode('utf-8')

plot_tfidf.scatter(x='x', y='y', source=tsne_df)
hover = plot_tfidf.select(dict(type=HoverTool))
hover.tooltips={"word": "@words"}
show(plot_tfidf)
user_17979260
Related