TypeError: the JSON object must be str, bytes or bytearray, not NoneType

Viewed 2018

When trying to translate the content of a column from NL to EN in a dataframe using the googletrans library, with the following code:

import pandas as pd
import numpy as np
from googletrans import Translator

#import newsarticles as dataframe
df_delpher = pd.read_csv (r'D:\file.csv')

#remove unnecessary column
df_delpher.drop(df_delpher.columns[0], axis=1, inplace=True)

#print number of rows
print(len(df_delpher.index))

#remove duplicate rows
df_delpher.drop_duplicates()

#re-print number of rows to check whether there where any duplicates.
print(len(df_delpher.index))

#In the case of the Delpher dataframe, no immediate duplicates where found, rows before and after are the same: 17284.

#remove first 6 charachters of the Date column so that only the year remains.
df_delpher['Date'] = df_delpher['Date'].str[6:]

#check the presence of every year in the dataframe to make sure all the years 1970-2018 are included
print(df_delpher.Date.unique())

#Years 1970 - 1995 are present in the Delpher dataframe

#count frequency of presence year in dataframe
print(df_delpher.Date.str.split(expand=True).stack().value_counts())

#count frequency of presence different papers in the datafrme
print(df_delpher.Newspaper.value_counts())
  
#remove all white spaces from content text
df_delpher.Content = df_delpher.Content.str.strip()


#remove rows with empty cells
    #newspaper
df_delpher['Newspaper'].replace('', np.nan, inplace=True)
df_delpher.dropna(subset=['Newspaper'], inplace=True)

    #Date
df_delpher['Date'].replace('', np.nan, inplace=True)
df_delpher.dropna(subset=['Date'], inplace=True)

    #Content
df_delpher['Content'].replace('', np.nan, inplace=True)
df_delpher.dropna(subset=['Content'], inplace=True)

#remove punctuation from content text
df_delpher['Content'] = df_delpher['Content'].str.replace(r'[^\w\s]+', '')

#find out what object the columns are
print(df_delpher['Content'].dtypes)

#translate dataframe:
    
translator = Translator()
df_delpher['Content_en'] = df_delpher['Content'].apply(translator.translate, src='nl', dest='en')

I receive the following error:

TypeError: the JSON object must be str, bytes or bytearray, not NoneType

Selecting a value in the first row of the column "Content" returns type string:

type(df_delpher['Content'][0])
Out[39]: str

The dataframe is based upon a csv file and dtype of the column returns an object.

The column does contain values as shown in the linked image: df

Sample data:

<bound method NDFrame.head of               Newspaper  Date  \
0                 Trouw  1970   
1          De Telegraaf  1970   
2                 Trouw  1970   
3         De Volkskrant  1970   
4          De Telegraaf  1970   
...                 ...   ...   
15088  Algemeen Dagblad  2005   
15089  Algemeen Dagblad  2005   
15090  Algemeen Dagblad  2005   
15091  Algemeen Dagblad  2005   
15092  Algemeen Dagblad  2005   

                                                 Content  
0      HENGELO  De Dikkersgroep verwacht voor 1969 ge...  
1      HENGELO dinsdag De Dlkkarsgroep verwacht oor I...  
2      DEN HAAG  De tendens tot concentratie die over...  
3      Van onze correspondent DEN HAAG 9 jan  De bouw...  
4      Van onze redactie wetenschappen AMSTERDAM vrij...  
...                                                  ...  
15088  ___ Wie zei tegen een roddelblad Ik ben dol op...  
15089  ___ Wie zei tegen een roddelblad Ik ben dol op...  
15090  ___ Wie zei tegen een roddelblad Ik ben dol op...  
15091  ___ Wie zei tegen een roddelblad Ik ben dol op...  
15092  ___ Wie zei tegen een roddelblad Ik ben dol op...  

[14026 rows x 3 columns]>

How can I overcome this JSon object error?

Thanks.

0 Answers
Related