How to upload any files to mongodb using Express, Multer, ES6, Router - without Moongoose?

Viewed 457

I would like to have my backend accept and upload files directly to mongodb. I am failing to achieve this using multer or formidable.

Problem 1: I am unable to reach the correct endpoint of my backend at all. All non-multipart/form-data endpoints are reachable and work properly. Problem 2: How to i properly configure it? What do i need to do in order to save file in memory then insert it into mongodb, preferrably without using moongoose (i do not need it so far).

Overview of project structure:

server.js
index.js
-api
--routes1.js
--routes2.js
...
-controllers
--route1.controller.js
--route2.controller.js
...
-dao
--route1DAO.js
--route2DAO.js
...
-middleware
--auth.js

Dependencies (package.json):

  "engines": {
    "node": "^14.17",
    "npm": "^6.14"
  },
  "dependencies": {
    "bcryptjs": "^2.4.3",
    "cors": "^2.8.5",
    "dotenv": "^10.0.0",
    "express": "^4.17.1",
    "jsonwebtoken": "^8.5.1",
    "mongodb": "^4.2.2",
    "multer": "^1.4.4"
  }

server.js:

import express from "express"
import cors from "cors"
import user from "./api/user.route.js"
import userprotected from "./api/userprotected.route.js"
import demo from "./api/demo.route.js"
import auth from "./middleware/auth.js"
import multer from "multer"

const app = express()

// Middleware
app.use(cors())
app.use(express.json())
app.use(express.urlencoded({extended: true}))
//app.use(multer) // not working --> Error: "requires middleware function"

// own public routes
app.use("/api/v1/user", user) 
// own private routes
app.use("/api/v1/userprot", auth, userprotected)
app.use("/api/v1/demo", auth, demo)
[...]
app.use("*", (req,res) => res.status(404).json({error: "not found"}))
export default app

index.js:

import app from "./server.js"
import mongodb from "mongodb"
import dotenv from "dotenv"
import UserDAO from "./dao/userDAO.js"
[...]

dotenv.config()
const MongoClient = mongodb.MongoClient

const port = process.env.PORT || 5000

MongoClient.connect(
    process.env.DEMO_DB_URI,
    {   
        maxPoolSize: 50,
        wtimeoutMS: 2500,
        useNewUrlParser: true
    }
    )
    .catch(err => {
        console.error(err.stack)
        process.exit(1)
    })
    .then(async client => {
        await UserDAO.injectDB(client)
        [...]
        app.listen(port, () => {
            console.log(`listening on port `+port)
        })
    })

The route i am trying to reach with postman:

import express from "express"
import DemoCtrl from "demo.controller.js"
const router = express.Router()
router.route("/test").post(DemoCtrl.uploadTest)
export default router

I am not reaching the handler:

export default class DemoController{
  static async uploadTest(req, res, next){
    try {
      console.log(req.file);
      res.status(200).json({status: "demo"})
    } catch (e) {
      res.status(500).json({error: e})
    }
  }
}

This is what i am trying to send from Postman, i triple checked headers and i keep runnig either into timeouts or into error 404. What i try to send from Postman

I tried to "Middleware"-ify the upload, however wherever i try to call it, it fails with a timeout error; I made sure the file i am trying to upload does not exceed the limit. This is the correspondig code.

import util from "util"
import Multer from "multer"
const maxSize = 2 * 1024 * 1024;

let processFile = Multer({
  storage: Multer.memoryStorage(),
  limits: { fileSize: maxSize },
}).single("file");


export default function(req, res){
  return util.promisify(processFile);   
}

I would be grateful for any hints or suggestions. I am clearly missing something.

1 Answers

There are many examples of how to use multer correctly, here is a quick working snippet of using it with express:

import express from "express";
const app = express();
const router = "import from file"
app.use("/api/v1", router);
const multer = require('multer');
const storage = multer.memoryStorage();
const upload = multer({storage: storage});

import {Router} from 'express';

const router = Router();

fucntion uploadTest(req, res) {
   const buffer = req.file.buffer;
   // do whatever you want with the buffer
}

router.post('/test', upload.single('file'), uploadTest);

export default router

So now regarding your problems.

Problem 1: There isn't much to say here, i'm not sure why you're "unable to reach the right route" but with my toy example you just have to send a post request to /api/v1/test/ and it should work.

Problem 2: I'm not sure why you'd want to save an entire file into Mongo, but if I were to do it I would just take the buffer, convert it to base62 and save it as a long string in some collection.

If you meant to use something like gridFS instead I personally don't have any experience with it, I choose to upload my files to s3.

Related