How can I manage that a JSON does not always have to be open?

Question

I have what I think is a bit more difficult "problem" to solve today. In my script, I have a function which reads out a corresponding translation/record from a JSON for each executed command.

The function looks like this:

response_string = {} # Gets the correct JSON from my JSON structure and reads out the correct message
def get_lang(guild, response):
    try:
        with open("languages.json", "r") as f: # Open the JSON to see which language the guild chose
            data = json.load(f) 

        return response_string[data[str(guild)]][response] # Get response from the corresponding JSON for the guild
    except:
        return response_string['english'][response] # If not set it is English

But here is the concern, that the JSON file is always open. This could become an issue in the end, I have been told. The JSON structure from which I get the responses looks like this:

./languages # Folder
./languages/german.json
./languages/english.json
...

In a JSON it looks like this:

{
    "all_langs": "Available Languages"
}

Say, for each language, there is a translation in each JSON. When running a command, I would then always call get_lang, "all_langs".

I was told that it would be smart to convert the JSON to a dict and then read out all the translations when starting the application, so you don't always have to have the JSON open. Is this a common or better practice?

I have already looked at the following articles:

I also started and thought about how to handle this.

client.languages_loaded = False # Set it to False 

@client.event
async def on_ready():
    print("Online")

    if client.languages_loaded:
        return
    
    client.languages_loaded = True

    client.languages = # Define all the JSON's here or start to store it in a dict?

But unfortunately I can't get any further, I'm more or less at the end with the knowledge, because in the end I don't know how the readout should always work without keeping the JSON open. About any tip I am grateful!

UPDATE: The code I use to get the correct language JSON is the following:

for i in os.listdir('./languages'):
    if i.endswith('.json'):
        with open(os.path.join('./languages', i)) as file: # Always open
            response = json.load(file) # Get the response from the correct JSON
        response_string[i.replace(".json", "")] = response # Give out response

user_17215197 · Answer

I think what you're trying to do would look something like this.

languages = {}

with open("languages.json", "r") as f: # Open the JSON
    languages = json.load(f)


response_string = {} # Gets the correct JSON from my JSON structure and reads out the correct message
def get_lang(guild, response):
    try:
        return response_string[languages[str(guild)]][response] # Get response from the corresponding JSON for the guild
    except:
        return response_string['english'][response] # If not set it is English

user_2819873 · Answer

If your json is not modified when the application run, a best practice should to have a object in memory to store this info ( a dataclass maybe). The only drawback of that is if the json files are huge it could take a lot of memory in this case.

Alternatively you can store the same data in a mysql database and instead of reading the file you just run a mysql request to get the translation. But I think as a good start having all translation in memory is better.

user_17668098 · Answer

If you want to want to modify your dictionary and not open it everytime just for reading, you can do something like this.

import json

bot = commands.Bot(...)

def startup_task():
   with open('languages.json()', 'r') as file:
        bot.langauges = json.load(file)

def modify_json():
    with open('languages.json', 'w') as file:
         json.dump(bot.languages, file)

@bot.command()
async def modifydict(ctx, key, value):
    bot.languages[key] = value # modifying it in memory first
    loop = bot.loop
    await loop.run_in_executor(None, modify_json) # in case json is too big and we want to avoid blocking 
    return await ctx.send(f"Updated the json successfully with key: `{key}` and value: `value`") # modified it in the json too


@bot.command()
async def readjson(ctx, key):
    try:
       value = bot.languages[key] # you are not reading json again!
    except KeyError:
       value = None
    if not value:
       return await ctx.send(f"No such key was found in dictionary")
    return await ctx.send(f"The value of key {key} was found to be {value}")

startup_task()
bot.run("TOKEN")

I don't fully understand what you exactly require but I think it's something along these lines? Let me kknow if I am wrong

How can I manage that a JSON does not always have to be open?

3 Answers