Remove comments from json file using Python

Mr_yassh 0
Viewed 89

I have a JSON file with comments shown below, I can't read the file in python as it is an invalid JSON file and I would like to have a pythonic way to remove all the lines in the file starting with /* as shown below:

/* 1 */
[{
    "_id" : ObjectId("abe"),
    "id" : "149",
    "objectType" : "act"
}
/* 2 */
{
    "_id" : ObjectId("abe415"),
    "id" : "449899009",
    "objectType" : "ity"
}]

I have tried the code below but getting the error in loads() of JSON: '''

import JSON

with open('data.json', 'r+',encoding='utf-8-sig') as handle:
    fixed_json = ''.join(line for line in handle if not line.startswith('/*'))
    final_data = json.loads(fixed_json)

print(final_data)

'JSONDecodeError: Expecting value: line 4 column 13 (char 16)'

Thanks in advance

json python
1 Answers

For your special input that you didn't share here, this is my solution with regex:

Note: you need to check your input errors and convert false, null, or other keywords to strings like "false".

import json
import re
import emoji
    

with open('tweets.json', 'r+') as handle:
   
    fixed_json = ''.join(line for line in handle)  
    # remove emojis
    fixed_json = emoji_pattern.sub(r'', fixed_json)
    fixed_json = emoji.replace_emoji(fixed_json, replace='')
    fixed_json = fixed_json.replace(' ', '')
    fixed_json = fixed_json.replace('false', '\"false\"')
    fixed_json = fixed_json.replace('null', '\"null\"')

rx = re.compile(r"\/\*.*\/([\n\s]*\{[\w\W]*?\}[\n\s]*)(?=\/\*.*\/)")

parts = rx.split(fixed_json) 

print(parts[0])
print(len(parts))


tweets=[]
parts = parts[1:]
for tweet in parts:
    
    tweet = tweet.replace('\udbb8\udf35\udbb8\udf38', '')
    tweet = emoji.replace_emoji(tweet, replace='')
    
    tweet = json.dumps(tweet)
    
    tweets.append(json.loads(tweet) )
print(len(tweets))

# Remove empty elements
result = []
for tweet in tweets:
    if len(tweet.strip())>0:
        result.append(tweet)
print(len(result))

# convert string to json
results = []
for res in result:
    try:
        results.append(eval(res))
    except:
        continue
print(len(results))

Output:

/*10000Tweets*/



19999
19998
10000
5022 #tweets that converted to JSON successfully
Shahab Rahnama0
Related