SQL Server query using knex crashes Nuxt app when results size is too large

Viewed 178

Using knex in a Nuxt app to query a SQL Server database hosted on Azure. When querying one particular table with ~150k rows, the app crashes but does print the length of the results returned. When querying a smaller table with ~2k rows, there is no problem.

Is there a limitation on how much data I can return from a single query? I need to be able to return about 1 million rows of data across several tables so that I can aggregate and display some calculations done with the raw table data.

I think it's understandable that the amount of data takes up too much memory, but I would like to know if there is any workaround to returning tons and tons of rows w/o issue.

api/routes/tickets.js

const { Router } = require('express');
const router = Router();
const knex_db = require('knex')({
    client: 'mssql',
    connection: {
        host: 'mydb.database.windows.net',
        user: 'user',
        password: 'secret',
        port: 1433,
        options: {
            database: 'mydatabase',
            encrypt: true
        }
    }
});

router.get('/tickets/all', async function(req, res) {

    const results = await knex_db('dbo.tickets');
    console.log('results.length: ' + results.length);
    res.json({data: results});
})

module.exports = router;

api/index.js

const express = require('express');

const app = express();

const tickets = require('./routes/tickets');

app.use(tickets);

module.exports = {
    path: '/api',
    handler: app
}

pages/setup/index.vue

<script>
    export default {
        async asyncData ({ $axios }) {
            const data = (await $axios.$get('/api/tickets/all')).data;
            // console.log(data);
            return { tickets: data }
        }
    }
</script>
1 Answers

I was able to resolve this issue by changing my code from

res.json(...)

to

res.status(200).json(...)

For some reason res.json must have been causing a memory leak or something of the sorts.

Related