I am having trouble appending dynamic data to my JSON file

projectmikey 0
Viewed 46

I have been having a bit of trouble appending new dynamic data to a JSON file. To sum up my project, I take in the projectName from an input form at the /new page. My API is then using the node.js's fs module to create a new JSON file with which I can then append the new dynamic data upon subsequential requests to my form. The variables are 1) projectName (is taken in from my form), 2) activeUser (which is programmed in through an environmental variable), 3) is the date of the request which I am acquiring through a timestamp variable with this function:

const timestamp = (JSON.parse(JSON.stringify(new Date())));

All three of these variables seem to print correctly for 2 subsequent requests and then on the third form submission there seems to be no new data appending to the JSON file. However i am relatively new to node.js and I can't seem to figure out where I am messing this up.

This is my API

pages/api/demos/index.js

import dbConnect from '../../../lib/dbConnect';
import Demo from '../../../models/Demo';
import fs from 'fs';

export default async function handler(req, res) {
  const {
    query: { id },
    method,
  } = req
  await dbConnect()

  switch (method) {
    case 'POST':
      try {
          //check if file exist
          if (!fs.existsSync('projects.json')) {
              //create new file if not exist
              fs.closeSync(fs.openSync('projects.json', 'w'));
          }
          // read file
          const timestamp = (JSON.parse(JSON.stringify(new Date())));
          const newFileName = req.body.projectName;
          const activeUser = process.env.ACTIVE_USERNAME;
          const file = fs.readFileSync('projects.json')
          const data = {
            "projects": [
                {
                    "username": activeUser,
                    "pages": [
                        {
                            "display": "routes",
                            "subpages": [
                                {
                                    "date": timestamp,
                                    "route": newFileName,
                                    "display": newFileName
                                }
                            ]
                        }
                    ]
                }
            ]
        }
          //check if file is empty
          if (file.length == 0) {
              //add data to json file
              fs.writeFileSync("projects.json", JSON.stringify([data]))
          } else {
              //append data to jso file
              const json = JSON.parse(file.toString())
              //add json element to json object
              json.push(data);
              fs.appendFileSync("projects.json", JSON.stringify(data))
          }

        const demo = await Demo.create(
          req.body
        )
        res.status(201).json({ success: true, data: demo })
      } catch (error) {
        res.status(400).json({ success: false })
      }
      break
    default:
      res.status(400).json({ success: false })
      break
  }
}

After the first form submission my JSON file projects.json looks like

[
    {
        "projects": [
            {
                "username": "projectmikey",
                "pages": [
                    {
                        "display": "routes",
                        "subpages": [
                            {
                                "date": "2022-09-12T19:03:09.547Z",
                                "route": "1",
                                "display": "1"
                            }
                        ]
                    }
                ]
            }
        ]
    }
]

and then after the 2nd form submission

[
    {
        "projects": [
            {
                "username": "projectmikey",
                "pages": [
                    {
                        "display": "routes",
                        "subpages": [
                            {
                                "date": "2022-09-12T19:03:09.547Z",
                                "route": "1",
                                "display": "1"
                            }
                        ]
                    }
                ]
            }
        ]
    }
]{
    "projects": [
        {
            "username": "projectmikey",
            "pages": [
                {
                    "display": "routes",
                    "subpages": [
                        {
                            "date": "2022-09-12T19:03:24.466Z",
                            "route": "2",
                            "display": "2"
                        }
                    ]
                }
            ]
        }
    ]
}

Oddly it seems to work for two form submissions and then the data stops appending to my file. This is after the third attempt, (no change to the file)

[
    {
        "projects": [
            {
                "username": "projectmikey",
                "pages": [
                    {
                        "display": "routes",
                        "subpages": [
                            {
                                "date": "2022-09-12T19:03:09.547Z",
                                "route": "1",
                                "display": "1"
                            }
                        ]
                    }
                ]
            }
        ]
    }
]{
    "projects": [
        {
            "username": "projectmikey",
            "pages": [
                {
                    "display": "routes",
                    "subpages": [
                        {
                            "date": "2022-09-12T19:03:24.466Z",
                            "route": "2",
                            "display": "2"
                        }
                    ]
                }
            ]
        }
    ]
}

It seems to stop working at all when I remove the pair of brackets around the initial JSON object. The line I am refering to is fs.writeFileSync("projects.json", JSON.stringify([data]))

I could really use another pair of eyes on this so I can see where I am messing this up! lol Thanks in advance for your time...

next.js reactjs json node.js javascript
1 Answers

Although it feels like you are "appending" to the file, you are actually doing something more complicated.

e.g. before state:

[ "one", "two" ]

desired after-state:

[ "one", "two", "three" ]

Notice that you can't just append text to the before-state JSON because there's already that pesky ] terminating the whole object.

Some failed attempts might look like:

failed attempt to append another entire array

[ "one", "two" ][ "three" ]

This is invalid because there are two root objects.

failed attempt to append just the rest of the array

[ "one", "two" ], "three" ]

That's no good either. The ] at the end of the original file needs to be overwritten or removed, so there's no way to just append. I suppose technically you could seek to the position of the final ] and then continue writing an incomplete object from there. But this is very awkward to remove the final ] from the source and to remove the initial [ from the chunk you're trying to append. It's just a difficult approach.

What you actually want to do is:

  1. read the entire JSON file
  2. parse the JSON into a JavaScript object (or create an empty object if the file didn't exist)
  3. Modify the JavaScript object as necessary (e.g. push into the array to add another element)
  4. stringify the JavaScript object into new JSON
  5. overwrite the entire file with the new JSON.

/* In Node.js:
const fs = require('fs');
try {
  initialJSON = fs.readFileSync('example.json');
} catch (ignore) {
  initialJSON = '[]';
}
*/
/* Mocked for this example: */
initialJSON = '["one","two"]';

// Common
obj = JSON.parse(initialJSON);
obj.push("three");
finalJSON = JSON.stringify(obj);

/* In Node.js:
fs.writeFileSync('example.json', finalJSON);
*/
/* Mocked for this example: */
console.log(finalJSON);

Wyck0
Related