What is the difference between "createReadStream" and "Readable" class?

user_13506965 0
Viewed 869

Can someone explaint Difference between createReadStream and readable in node.js? By my observes they are similar, so what is under the hood difference, and when should each be used?

for example

const s3 = new AWS.S3({
      accessKeyId: AWSConfig.AWSConfig.ACCESS_KEY_ID,
      secretAccessKey: AWSConfig.AWSConfig.SECRET_ACCESS_KEY,
    });

    const params = {
      Bucket: AWSConfig.AWSConfig.bucket,
      Key: "somebucketName/1620072325205",
    };
    const file = await s3.getObject(params).promise();
    const fileSize = file.ContentLength / (1024 * 1024);

    const read = new Readable({
      read(fileSize) {
        this.push(file.Body);
        this.push(null);
      },
    });

    read.pipe(res);

this is similar to

const s3 = new AWS.S3({
      accessKeyId: AWSConfig.AWSConfig.ACCESS_KEY_ID,
      secretAccessKey: AWSConfig.AWSConfig.SECRET_ACCESS_KEY,
    });

    const params = {
      Bucket: AWSConfig.AWSConfig.bucket,
      Key: "somebucketName/1620072325205",
    };
    const file = await s3.getObject(params).createReadStream();
    file.pipe(res)
node.js javascript
2 Answers

In a NodeJS, you can create a readable stream in a few ways:

SOLUTION 1

You can do it with fs module. The function fs.createReadStream() allows you to open up a readable stream and all you have to do is pass the path of the file to start streaming in.

const fs = require('fs');

const readable_stream = fs.createReadStream('file_path');

SOLUTION 2

If you don't want to create file, you can create an in-memory stream and do something with it (for example, upload it somewhere). ​You can do this with stream module. You can import Readable from stream module and you can create a readable stream. When creating an object, you can also implement read() method which is used to read the data out of the internal buffer. If no data available to be read, null is returned. The optional size argument specifies a specific number of bytes to read. If the size argument is not specified, all of the data contained in the internal buffer will be returned.

const Readable = require('stream').Readable;

const readable_stream = new Readable({
  ​read(size) {
   ​// ...
​  }
});

SOLUTION 3

When you are fetching something over the network, that can be fetched like stream (for example you are fetching a PDF document from some API).

const axios = require('axios');

const readable_stream = await axios({
  method: 'get',
  url: "pdf_resource_url",
  responseType: 'stream'
}).data;

SOLUTION 4

Third party packages can support creating of streams as a feature. That is a way with aws-sdk package from your example.

SUMMARIZE AND CONCLUSION

You can create a readable stream in a few ways. Since you are already using aws-sdk package, I would say that you should go with using their createReadStream(), instead of importing stream module and creating readable stream with it.

user_143898300

Another solution that worth to mention is the Readable.from method which allows you to create a reable stream by providing an iteratable to it (a String for example). See the following example:

const { Readable } = require("stream");

const readableStream = Readable.from("In-memory data to a readable stream");
user_89300250
Related