How can I use papaparse on Node JS?

user_13529025 0
Viewed 2853

I have been trying to get papaparse running on nodejs for a very long time now and I keep failing. I want to try to load my local CSV and then give this papaparse? How to do that? My code does not work.

import papa from "papaparse";
import fs from "fs";

export const convertCSV = async (res: Response) => {
  const file = await fs.createReadStream("../files/test.csv");
  papa.parse(file, {
    header: true,
    complete: function (results, file) {
      console.log("Complete", results.data.length, "records.");
    },
  });
};

results.data.length is always 0.

My CSV file is located in the files folder, which is located in the src folder:

src/files/test.csv
papaparse typescript node.js csv javascript
3 Answers

As Erick says in their comment, the download option is not supported by Node. However, it does tell us the Node interface accepts a ReadableStream instead of a File.

This works for me:

Papa.parse(fs.createReadStream("sample/result.csv"), {
    complete: function(results) {
        console.log(results["data"]);
    }
});
user_14215550

From the Papaparse README, it supports Node in this way:

Papa Parse can parse a Readable Stream instead of a File when used in Node.js environments (in addition to plain strings). In this mode, encoding must, if specified, be a Node-supported character encoding. The Papa.LocalChunkSize, Papa.RemoteChunkSize, download, withCredentials and worker config options are unavailable.

Papa Parse can also parse in a node streaming style which makes .pipe available. Simply pipe the Readable Stream to the stream returned from Papa.parse(Papa.NODE_STREAM_INPUT, options). The Papa.LocalChunkSize, Papa.RemoteChunkSize, download, withCredentials, worker, step and complete config options are unavailable. To register a callback with the stream to process data, use the 'data' event like so: stream.on('data', callback) and to signal the end of stream, use the 'end' event like so: stream.on('end', callback).

user_4244980

Your code is fine, the problem is in the file path. fs.createReadStream needs an absolute path, or a path that's relative to the root of your project.

Assuming this is the project structure:

files/
  - test.csv
src/
  - convert-csv.ts
package.json
tsconfig.json

Then you can use a path that's relative to the root: './files/test.csv', or if you really want to use a path that's relative to the file then you can use path.resolve(__dirname, '..', 'files', 'test.csv').

See NodeJS docs for path.resolve, it will use the path segments to resolve an absolute path for you.

user_141432140
Related