How can I insert JSON efficiently without looping through all items in the array?

user_8558723 0
Viewed 49

I have 3 JSON files. Each contain 10,000 objects each with 2 properties.

I have been trying to take a key value from each file, and use it to create a new JSON file, under a slightly different structure. My code works, but it takes a really long time to complete via my current code.

The issue is caused by a for each loop inside of a for each loop. I am looking for the "TokenId" of all items to check I am inserting in the correct location of the JSON. So, to begin with token 0, starts looping through the new JSON, finds it straight away. token 1 then checks token 0, it's not a match, so then checks token 1 which is. As I started to get to the higher numbers, for example 4000, it is checking every item 0-3999, which just seems unnecessary and incredibly time and resource inefficient. How else can I do this?

Here's the code:

            dynamic objFinal = JObject.Parse("{ 'Tokens':[] }");
            JArray finalArray = (JArray)objFinal["Tokens"];
            DirectoryInfo dir = new DirectoryInfo(@"C:\Users\s\Desktop\JsonTesting\");
            foreach (var file in dir.GetFiles("*.json"))
            {
                dynamic objJson = JObject.Parse(File.ReadAllText(file.FullName));
                JArray tokens = (JArray)objJson["Tokens"];
                foreach (JToken token in tokens)
                {
                    JToken searchResult = finalArray.Where(t => Convert.ToString(t["TokenId"]) == Convert.ToString(token["TokenId"])).FirstOrDefault();
                    if (searchResult == null)
                    {
                        //push new tokenid
                        
                        JObject newFileRanks = new JObject();
                        newFileRanks[file.Name.Remove(file.Name.Length - 5)] = token["rank"];
                        dynamic newToken = new JObject();
                        newToken["TokenId"] = token["TokenId"];
                        newToken["Ranks"] = newFileRanks;
                        finalArray.Add(newToken);
                        Console.WriteLine("Added new token");
                    }
                    else
                    {
                        //append to fileRanks

                        JObject newFileRanks = new JObject();
                        newFileRanks[file.Name.Remove(file.Name.Length - 5)] = token["rank"];

                        foreach (var item in finalArray)
                        {
                            Console.WriteLine("Current token TokenId: " + token["TokenId"]);
                            Console.WriteLine("Current Item TokenId: " + item["TokenId"]);
                            if ((string)item["TokenId"] == (string)token["TokenId"])
                            {
                                Console.WriteLine("Adding Rank");
                                dynamic ranksObject = item["Ranks"];
                                ranksObject[file.Name.Remove(file.Name.Length - 5)] = token["rank"];
                                break;
                            }

                        }
                    }
                }
            }
            Console.Write(JsonConvert.SerializeObject(objFinal));
            File.AppendAllText(@"C:\Users\samcl\Desktop\JsonTesting\cryptohippos_Rankings.json", JsonConvert.SerializeObject(objFinal));`

Here is the raw JSON structure for each file:

{"Tokens":[{"TokenId":0,"rank":2804},{"TokenId":1,"rank":977},

And here is the desired result JSON structure:

{"Tokens":[{"TokenId":0,"ranks":{"File1rank":977},{"File2rank":987},{"File3rank":967}},{"TokenId":1,"ranks":{"File1rank":977},{"File2rank":987},{"File3rank":967}}}
sorting arrays json c#
2 Answers

Deserialize the JSON to an object and try manipulating the object using Linq functions.

Serialize back to JSON when done.

JSON objects are meant for data transfer and should ideally be used only for that purpose.

user_147238980

You should use a proper object model to do this, rather than JObject and dynamic etc, all of which can be slow.

There are other efficiencies, for example you shouldn't loop through your existing list each time. Instead store your intermediate Token objects in a dictionary so you can quickly access it.

class Root
{
    public List<Token> Tokens = new List<Token>();
}

class Token
{
    public int TokenId;
    public int? rank;
    public Dictionary<string, int> ranks;
}

var rootObj = JsonConvert.DeserializeObject<Root>(File.ReadAllText(file.FullName));

var tokensDict = new Dictionary<int, Token>();

DirectoryInfo dir = new DirectoryInfo(@"C:\Users\s\Desktop\JsonTesting\");

foreach (var file in dir.EnumerateFiles("*.json"))
{
    foreach (var token in rootObj.Tokens)
    {
        if(!tokensDict.TryGetValue(token.TokenId, out var dictToken)
        {
            tokensDict[token.TokenId] = dictToken = token;
            token.ranks = new Dictionary<string, int>();
            token.ranks.Add(file.Name.Remove(file.Name.Length - 5), token.rank);
            token.rank = null;
        }
        else
        {
            dictToken.ranks.Add(file.Name.Remove(file.Name.Length - 5), token.rank);
        }
    }
}

var objFinal = new Root();
objFinal.Tokens.AddRange(tokensDict.Value);

File.AppendAllText(@"C:\Users\samcl\Desktop\JsonTesting\cryptohippos_Rankings.json", JsonConvert.SerializeObject(objFinal));`
user_148689970
Related