Can I upload data to only part of a json file?

user_15919789 0
Viewed 87

I am wondering if there is any way to update the json file without having to completely rewrite the json file. As shown in the example below I first had to retrieve all the data from the json file then add to/ edit the original data then rewrite the new data dictionary over the original file. Is it possible to just edit a few lines of the json file directly?

I have a main file and a json file that look as follows:

MAIN FILE

import json

with open('animals.json') as f:
    data = json.load(f)

attributes = {"Food": "", "Size": "", "Habitat": ""}

data["Animal 1"]["Food"] = "Meat"
data["Animal 1"]["Size"] = "Medium"
data["Animal 1"]["Habitat"] = "Savannah"

attributes["Food"] = "Nuts"
attributes["Size"] = "Small"
attributes["Habitat"] = "Forest"
data["Animal 3"] =  dict(attributes)

# attributes["Food"] = "Plankton"
# attributes["Size"] = "Large"
# attributes["Habitat"] = "Ocean"
# data["Animal 2"] =  dict(attributes)

with open('animals.json', 'w') as f:
    json.dump(data, f, indent=4)

JSON FILE

{
    "Animal 1": {
        "Food": "Banana",
        "Size": "Medium",
        "Habitat": "Jungle"
    },
    "Animal 2": {
        "Food": "Plankton",
        "Size": "Large",
        "Habitat": "Ocean"
    }
}

Currently the code updates the json file (Animal 1 attributes and adds Animal 3) to look like this:

{
    "Animal 1": {
        "Food": "Meat",
        "Size": "Medium",
        "Habitat": "Savannah"
    },
    "Animal 2": {
        "Food": "Plankton",
        "Size": "Large",
        "Habitat": "Ocean"
    },
    "Animal 3": {
        "Food": "Nuts",
        "Size": "Small",
        "Habitat": "Forest"
    }
}

Are there any alternatives to this method?

dictionary file python json
1 Answers

I presume you'll scale up the process with much more data, so this json data overwritting can be easily achieve by using pandas dataframes. The main thing here is that you don't have to worry about each json field being updated as you showed with your code. I won't go through saving/writing file thing, as it's not the problem. If not for the json info, the solution takes up to only 6 lines of code with this approach. The step-by-step explanation I show after the full code.

By the way, you can also read from json file directly by using pd.read_json('file_name.json') and writing to it with datraframe_object.to_json('file_name.json').

Full code for those who want to reproduce:

import json
import pandas as pd

original_json = {
    "Animal 1": {
        "Food": "Banana",
        "Size": "Medium",
        "Habitat": "Jungle"
    },
    "Animal 2": {
        "Food": "Plankton",
        "Size": "Large",
        "Habitat": "Ocean"
    }
}

overwrite_json = {
    "Animal 1": {
        "Food": "Meat",
        "Size": "Medium",
        "Habitat": "Savannah"
    },
    "Animal 2": {
        "Food": "Plankton",
        "Size": "Large",
        "Habitat": "Ocean"
    },
    "Animal 3": {
        "Food": "Nuts",
        "Size": "Small",
        "Habitat": "Forest"
    }
}

#here the six lines to get your updated json info
original_frame = pd.DataFrame(original_json).T
overwrite_frame = pd.DataFrame(overwrite_json).T
original_frame['index'] = original_frame.index
overwrite_frame['index'] = overwrite_frame.index

updated_frame = original_frame.merge(overwrite_frame,how='right').set_index('index')
final_json = updated_frame.to_json(orient='index',indent=4)
print(final_json)

The output:

{
    "Animal 1":{
        "Food":"Meat",
        "Size":"Medium",
        "Habitat":"Savannah"
    },
    "Animal 2":{
        "Food":"Plankton",
        "Size":"Large",
        "Habitat":"Ocean"
    },
    "Animal 3":{
        "Food":"Nuts",
        "Size":"Small",
        "Habitat":"Forest"
    }
}

Step-by-step explanation:

  1. Import pandas, which is commonly used for data science, so it's well documented and used by many.
import json
import pandas as pd
  1. Here I assume you'll do your 'with open' file thing. For simplicity I'll just assign the json objects with the info you provided.
original_json = {
    "Animal 1": {
        "Food": "Banana",
        "Size": "Medium",
        "Habitat": "Jungle"
    },
    "Animal 2": {
        "Food": "Plankton",
        "Size": "Large",
        "Habitat": "Ocean"
    }
}

overwrite_json = {
    "Animal 1": {
        "Food": "Meat",
        "Size": "Medium",
        "Habitat": "Savannah"
    },
    "Animal 2": {
        "Food": "Plankton",
        "Size": "Large",
        "Habitat": "Ocean"
    },
    "Animal 3": {
        "Food": "Nuts",
        "Size": "Small",
        "Habitat": "Forest"
    }
}
  1. Create pandas dataframes of the json information. I transposed (.T) here to keep your original json records orientation.
original_frame = pd.DataFrame(original_json).T
overwrite_frame = pd.DataFrame(overwrite_json).T

The dataframes look like this. They're indexed by 'Animal 1', 'Animal 2' and so on:

print(original_frame)
            Food    Size   Habitat     
Animal 1    Banana  Medium  Jungle  
Animal 2  Plankton   Large   Ocean

print(overwrite_frame)
             Food    Size   Habitat     
Animal 1      Meat  Medium  Savannah  
Animal 2  Plankton   Large     Ocean  
Animal 3      Nuts   Small    Forest
  1. Store the indexes of each dataframe in a new column for later use.
original_frame['index'] = original_frame.index
overwrite_frame['index'] = overwrite_frame.index
#example of 'index' column created
print(original_frame)
           Food    Size    Habitat     index
Animal 1    Banana  Medium  Jungle  Animal 1
Animal 2  Plankton   Large   Ocean  Animal 2
  1. Then we merge the two dataframes, overwritting the original_frame with the overwrite_frame. See that I use how='right' to give preference for the overwrite_frame and set_index to use the previously saved index columns.
updated_frame = original_frame.merge(overwrite_frame,how='right').set_index('index')
print(update_frame)
             Food    Size   Habitat
index                               
Animal 1      Meat  Medium  Savannah
Animal 2  Plankton   Large     Ocean
Animal 3      Nuts   Small    Forest
  1. Then simply generate a new json from the resulting dataframe
final_json = updated_frame.to_json(orient='index',indent=4)
print(final_json)
{
    "Animal 1":{
        "Food":"Meat",
        "Size":"Medium",
        "Habitat":"Savannah"
    },
    "Animal 2":{
        "Food":"Plankton",
        "Size":"Large",
        "Habitat":"Ocean"
    },
    "Animal 3":{
        "Food":"Nuts",
        "Size":"Small",
        "Habitat":"Forest"
    }
}

If I got your question right, this solves your bottleneck!

user_160039190
Related