I am a prospective graduate student studying deep learning.
I ran the code below on my macbook. The result was success. All json files are loaded and processed well.
I figured it would be faster if I processed the data with a lab computer with a bigger 16gb ram.
However, it was not executed in Windows as a Memory Error occurred.
I set Windows virtual memory paging size and Pycharm vm size following advice on the internet. but error was not solved.

virtual memory paging file size setting

how to run the code below in Windows Desktop? and why the code does not run in the Desktop though it has bigger RAM?

Settings

My computer settings are as follows:

Individual Notebook

• m1 silicon Macbook Air
• 8gb ram
• 256gb storage

Desktop in Lab

• Windows 10 Education
• 16gb ram
• 256gb ssd, 1TB hdd storage

requirements.txt

• python=3.8.12
• pysimdjson=4.0.3
• pandas=1.3.5

json file

• NXLS~.json (2.39gb)
• SXLS~.json (1.06gb)

Code

my codes are as follows:

import simdjson
import pandas as pd
from tqdm import trange


if __name__ == '__main__':
    # json key names corresponding to column names
    mapping_key = ['doc_index', 'begin', 'end', 'word_id', 'word', 'pos', 'sense_id']         
    # dataframe column name
    columns = ['sentence', 'target_index_start', 'target_index_end', 'target_word_id', 'target_word', 'target_pos', 'sense_id']
    corpus = pd.DataFrame(columns=columns)

    # read .json dataset.
    data_path = './dataset'
    NXLS = 'NXLS2002104060.json'
    SXLS = "SXLS2002104060.json"

    with open(data_path + '/' + NXLS, "rb") as f:
        nxls_doc = simdjson.loads(f.read())


    with open(data_path + '/' + SXLS, "rb") as f:
        sxls_doc = simdjson.loads(f.read())

    # parsing
    sentences = []
    WSDs = []
    for i in trange(len(nxls_doc["document"])):
        for j in range(len(nxls_doc["document"][i]["sentence"])):
            sentence = nxls_doc["document"][i]["sentence"][j]["form"]
            sentences.append(sentence)
            wsd = nxls_doc["document"][i]["sentence"][j]["WSD"]
            for k in range(len(wsd)):
                wsd[k]["doc_index"] = j
            # wsd[(k for k in range(len(wsd)))]["doc_index"] = j
            WSDs.append(wsd)

    for i in trange(len(sxls_doc["document"])):
        for j in range(len(sxls_doc["document"][i]["sentence"])):
            sentence = sxls_doc["document"][i]["sentence"][j]["form"]
            sentences.append(sentence)
            wsd = sxls_doc["document"][i]["sentence"][j]["WSD"]
            for k in range(len(wsd)):
                wsd[k]["doc_index"] = j
            # wsd[(k for k in range(len(wsd)))]["doc_index"] = j
            WSDs.append(wsd)

    # list -> Dataframe (restructure)
    for i in trange(len(WSDs)):
        wsd = WSDs[i]
        for j in range(len(wsd)):
            row = []
            doc_index = wsd[j]["doc_index"]
            row.append(sentences[doc_index])
            #for k in range(len(columns)-1):
            row.extend([wsd[j][mapping_key[k+1]] for k in range(len(columns)-1)])
            #corpus[columns[k+1]] = wsd[j][mapping_key[k+1]]
            corpus.loc[len(corpus)] = row

Result

Mac
result1 in m1 silicon

result2 in m1 silicon. because of the speed of processing, I stopped it.

json successfully loaded in m1 silicon

windows
result in Windows 10