jsonDiff in a loop for lots of files

user_3801167 0
Viewed 357

The goal is to get the difference of around 30'000 json files in 2 two folders. In both Folders their names are just numbers (1.json, 2.json etc.). So basically get the diff of the two folders.

I'm using the jsonDiff module to get the difference which works fine for single files but i haven't found a way to use the module in a loop.

f = open("/folder1/1.json")
fe = open("/folder2/1.json")
jyson1 = json.load(f)
jyson2 = json.load(fe)

print(diff(jyson1, jyson2))

EDIT: Because i have to parse through 2 folders, I think i need 2 for loops with glob like this

for f in glob("/folder1/*.json"):
    with open(f) as fe:
        data_old = json.load(fe)

for e in glob("/folder2/*.json"):
    with open(e) as ee:
        data_new = json.load(fe)

I struggle to understand how i can use the diff() method here

diff python json
1 Answers

Here is a code snippet from which you can work on. It also only tests for files which are common in both folders (skips files which are missing in one folder for any reason). In contrast to your code snippet you need to work in one not in two for loops:

import os

names1 = os.listdir("folder1") # names of all json files in folder1
names2 = os.listdir("folder2") # names of all json files in folder2
commonnames = set(names1) & set(names2) # set of all file names which are in folder1 and folder2

for filename in commonnames:
  json1 = json.load(open(os.path.join("folder1", filename))
  json2 = json.load(open(os.path.join("folder2", filename))

  print(diff(json1, json2))
user_11651550
Related