So I have an all text no numeric data file in English, Russian, etc. languages mixed. I imported google translator to solve this issue but it only translated one of my column and sometimes shows connection timed out error. The data file is quite large. Here is my code:
Train = pd.read_json('Train.jsonl',lines=True)
Train
train = Train.copy()
from googletrans import Translator
translator = Translator()
train.rename(columns=lambda x: translator.translate(x).text, inplace=True)
train.columns
translations = {}
for column in train.columns:
# unique elements of the column
unique_elements = train[column].unique()
for element in unique_elements:
# add translation to the dictionary
translations[element] = translator.translate(element).text
print(translations)
(CONNECTION TIMED OUT HERE BUT MY INTERNET IS FINE).
How to resolve this issue?
- Also, I am using this all text database. I want to write an algorithm using my train data to test whether sentences have something in common. Should I convert my data to float and the use Decision Tree Classifier? or linear regression or what?
P.S I'm very new to coding so sorry if this question sounds vague or something else.