Queing scraping jobs with Bull, Puppeteer, NodeJS and Express

I've recently built a platform on Heroku to host Puppeteer scraping scripts (Express, Node JS). Was going great until the scripts started taking longer and longer to complete. Per Heroku's documentations, anything longer then 30 seconds times out the request and it fails.

The way around this is by using Redis server (specifically with the npm module Bull).

I've never heard of queueing before so this is going to be a learning opportunity for myself.

Below is my current code (which works).

I will run you through the process (thoroughly):

1. The server gets the POST request from the client asking it to run this script stored in the database:

router.post(
  "/get-data/:id", // route
  ensureAuth, // ensure the person making the request is allowed to make the request in the first place
  fsController.createFile, // creates a .js file where the script will be copied to
  scraperController.getData, // runs the function in the newly created .js file
  csvController.buildCSVAndDeliver // adds that data to a csv file and downloads it
);

2. The file is created and the code from the database is copied to it

const Script = require("../models/Script");
const fs = require("fs");

exports.createFile = async (req, res, next) => {
  try {
    const script = await Script.findById({ _id: req.params.id });
    let filename = `./scraper/${script._id}.js`;

    // While loop to check if file exists and to append a digit if it does.
    let counter = 0;
    while (fs.existsSync(filename)) {
      filename = `./scraper/${script._id}${counter}.js`;
      counter++;
    }

    fs.writeFileSync(filename, script.script);
    req.filename = filename;
    next();
  } catch (error) {
    fs.unlink(filename);
    res.send(error, "Please Try Again");
    res.redirect('/dashboard')
  }
};

3. The scraperController takes that file and executes it

const fs = require("fs");

exports.getData = async (req, res, next) => {
  try {
    let scraper = require(`.${req.filename}`);
    const response = await scraper(req.body);
    req.response = response;
    fs.unlinkSync(req.filename);
    next();
  } catch (error) {
    res.send(error, "Please Try Again");
    fs.unlinkSync(req.filename);
  }
};

4. Finally, the CSV file is downloaded

const fs = require("fs");
const { Parser } = require("json2csv");

exports.buildCSVAndDeliver = async (req, res) => {
  let csvFile = `./csv/${req.params.id}.csv`;

  try {
    const parser = new Parser();
    const csv = parser.parse(req.response);
    fs.writeFileSync(csvFile, csv);
    res.download(csvFile, function () {
      fs.unlinkSync(csvFile);
    });
  } catch (error) {
    console.log(error);
  }
}

Now that you know the process, where do I add the queing code? Would it even work with this code structure?

Thank you, and should you need additional information from me, please don't hesitate to ask.