JSON.parse() causes error: `SyntaxError: Unexpected token in JSON at position 0`

To further explain @Luillyfe's answer:

Ah-ha! fs.readFileSync("data.json") returns a Javascript object!

Edit: Below is incorrect...But summarizes what one might think at first!

I had through that was a string. So if the file was saved as UTF-8/ascii, it would probably not have an issue? The javascript object returned from readFileSync would convert to a string JSON.parse can parse? No need to call JSON.stringify?

I am using powershell to save the file. Which seems to save the file as UTF-16 (too busy right now to check). So I get "SyntaxError: Unexpected token � in JSON at position 0."

However, JSON.stringify(fs.readFileSync("data.json")) correctly parses that returned file object into a string JSON can parse.

Clue for me is my json file contents looking like the below (after logging it to the console):

�{ " R o o m I D _ L o o k u p " :   [
         {
                 " I D " :     1 0 ,
                 " L o c a t i o n " :     " f r o n t " ,
                 " H o u s e " :     " f r o n t r o o m "
         }
}

That doesn't seem like something a file would load into a string...

Incorrect being (this does not crash...but instead converts the json file to jibberish!):

const jsonFileContents = JSON.parse(JSON.stringify(fs.readFileSync("data.json")));

I can't seem to find this anywhere. But makes sense!

Edit: Um... That object is just a buffer. Apologies for the above!

Solution:

const fs = require("fs");

function GetFileEncodingHeader(filePath) {
    const readStream = fs.openSync(filePath, 'r');
    const bufferSize = 2;
    const buffer = new Buffer(bufferSize);
    let readBytes = 0;

    if (readBytes = fs.readSync(readStream, buffer, 0, bufferSize, 0)) {
        return buffer.slice(0, readBytes).toString("hex");
    }   

    return "";
}

function ReadFileSyncUtf8(filePath) {
    const fileEncoding = GetFileEncodingHeader(filePath);
    let content = null;

    if (fileEncoding === "fffe" || fileEncoding === "utf16le") {
        content = fs.readFileSync(filePath, "ucs2"); // utf-16 Little Endian
    } else if (fileEncoding === "feff" || fileEncoding === "utf16be") {
        content = fs.readFileSync(filePath, "uts2").swap16(); // utf-16 Big Endian
    } else {
        content = fs.readFileSync(filePath, "utf8");
    }

    // trim removes the header...but there may be a better way!
    return content.toString("utf8").trimStart();
}

function GetJson(filePath) {
    const jsonContents = ReadFileSyncUtf8(filePath);
    console.log(GetFileEncodingHeader(filePath));

    return JSON.parse(jsonContents);
}

Usage:

GetJson("data.json");

Note: I don't currently have a need for this to be async yet. Add another answer if you can make this async!

As mentioned by TamusJRoyce I ended up using the util.TextDecoder class to come up with a robust way to read both UTF-8 (without BOM) and UTF-8 (with BOM). Here's the snippit, assuming that file input.json is UTF-8 (with or without BOM) and contains valid JSON.

const fs = require('fs');
const util = require('util');

const rawdata = fs.readFileSync('input.json');
const textDecoder = new util.TextDecoder('utf-8');
const stringData = textDecoder.decode(rawdata);
const objects = JSON.parse(stringData);

const fs = require('fs');
const myConsole = new console.Console(fs.createWriteStream('./output.json'));
myConsole.log(object);

This will create an output file with all the output which can been triggered through console.log(object).

This is the easiest way to convert the console.log() output into a file.`