json.decoder.JSONDecodeError: While Parsing data form an API contains Japanese Characters

user_12993150 0
Viewed 55

I was trying to parse an API response with Scrapy and JSON but it's generating me an error json.decoder.JSONDecodeError:

I tried all the Japanese encoding types from here but none worked. Hope there is a solution from you guys.

Below is my code:

import scrapy
import json

class MakuakeSpider(scrapy.Spider):
    name = 'makuake'
    start_urls = ['https://api.makuake.com/v2/projects?page=1&per_page=15&category_code=technology']

    def parse(self, response):
        json_rsp = json.loads(response.body).encode("encoding-type-jps").decode("utf-8")
        print(json_rsp)
scrapy python json
1 Answers

try this response.text

import scrapy
import json

class MakuakeSpider(scrapy.Spider):
    name = 'makuake'
    start_urls = ['https://api.makuake.com/v2/projects?page=1&per_page=15&category_code=technology']

    def parse(self, response):
        json_rsp = json.loads(response.text)
        print(json_rsp)
user_48351220
Related