Here's a python script that contains a nested loop that goes through a list of dictionaries and then performs a string operation on each key in the dictionary. The number of keys to process is ~30,000 and each takes between 2-3 seconds according to profiling.
Here's an example of the code in question:
def json_list(incoming_dict) -> List[Dict[str, Any]]:
dictionaries: List[Dict[str, Any]] = []
num_rows: int = len(incoming_dict["data"])
print("number of data rows: ", num_rows)
flattened_dictionary: Dict[str, str] = flatten(incoming_dict,
root_keys_to_ignore={"timestamp", "status_code", "status_message"})
for index, _json in enumerate(flattened_dictionary):
print("number of flat json keys: ", len(flattened_dictionary))
working_json = json.loads(json.dumps({"timestamp": "2022-01-22", "status_code": "200", "status_message": "Success"}))
for key, value in flattened_dictionary.items():
data_prefix = f"data_{index}_"
if key.startswith(data_prefix):
working_json[key.replace(data_prefix, "")] = value
dictionaries.append(working_json)
return dictionaries
Is it possible to write a Rust script to drop in and handle this more efficiently. If so, how would it be done so it can be dropped into the Python script?