How to write Arabic to a CSV file

user_9453429 0
Viewed 719

I am trying to extract tweets with Python and store them in a CSV file, but I can't seem to include all languages. Arabic appears as special characters.

def recup_all_tweets(screen_name,api):
    all_tweets = []
    new_tweets = api.user_timeline(screen_name,count=300)
    all_tweets.extend(new_tweets)
    #outtweets = [[tweet.id_str, tweet.created_at, tweet.text,tweet.retweet_count,get_hashtagslist(tweet.text)] for tweet in all_tweets]
    outtweets = [[tweet.text,tweet.entities['hashtags']] for tweet in all_tweets]
  #  with open('recup_all_tweets.json', 'w', encoding='utf-8') as f:
   #     f.write(json.dumps(outtweets, indent=4, sort_keys=True))

    with open('recup_all_tweets.csv', 'w',encoding='utf-8') as f:
       writer = csv.writer(f,delimiter=',')
       writer.writerow(["text","tag"])
       writer.writerows(outtweets)
   # pass

    return(outtweets)
twitter csv python
2 Answers

Example of writing both CSV and JSON:

#coding:utf8
import csv
import json

s = ['عربى','عربى','عربى']

with open('output.csv','w',encoding='utf-8-sig',newline='') as f:
    r = csv.writer(f)
    r.writerow(['header1','header2','header3'])
    r.writerow(s)

with open('output.json','w',encoding='utf8') as f:
    json.dump(s,f,ensure_ascii=False)

output.csv:

header1,header2,header3
عربى,عربى,عربى

output.csv viewed in Excel:

enter image description here

output.json:

["عربى", "عربى", "عربى"]

Note Microsoft Excel needs utf-8-sig to read a UTF-8 file properly. Other applications may or may not need it to view properly. Many Windows applications required a UTF-8 "BOM" signature at the start of a text file or will assume an ANSI encoding instead. The ANSI encoding varies depending on the localized version of Windows used.

user_2356980

Maybe try with f.write(json.dumps(outtweets, indent=4, sort_keys=True, ensure_ascii=False))

user_84787000
Related