mean.io stack angular long running operations with http request

Viewed 501

I am using the mean stack do develop a web application that performs several operations over several mongo db collections.

The systems overal intention is to perform data sinchronization between several systems. Each system data is stored in a different collections. Needless to say that these collection's documents have properties in common.

At the nodeJS, or the API level, I created get and post requests that will loop over the main collection asynchronouslye and will try to link documents on that collection with documents in other collections. Naturally this process takes for several minutes.

The documents that were possible to link are stored in a different mongodb collection. These will be later processed in another node js API call.

After the API is set I am creating a controller and a service with angular in order to connect to the api and perform the api get and post requests.

When I get to this point in the UI where I click in a button to perform this link operation, the operation starts without blocking the browser but eventually the browser get an ERR_EMPTY_RESPONSE because the operation is still undergoing so I never actually call the response.end().

What I thought could be a possible solution:

  1. reply back as soon as the operation starts: PROBLEM -> I need to make sure this operation finishes in order to make other calls on the resulting collection. So I did this operation to finishs to allow others to be done on the resulting data set.

What can I do in order to avoid the browser to close the connection before the long running operation in the background is completed?

Code:

html-> Button which the user clicks to start linking records:

<div>
    <md-button aria-label="link all records" ng-click="vm.linkRecords()">
        <md-icon class="mdi mdi-plus"></md-icon>Link Records
    </md-button>
</div>

controller: the canLink, canGrade are all boolean variables so that When the server is undergoing one operation none of the others can be performed. Once the operation finishes, it returns true and all operations are allowed.

function linkRecords() {
    $state.go('app.records.link');
}// END linkRecords page

$scope.$on('linkRecords', function(event, batchSize){
    if ( canLink && canGrade) {
        showToast("Started linking records");
        canLink  = 0;
        canGrade = 0;
        recordService.linkRecords(batchSize).then(function(finished){
            canLink     = finished;
            canGrade    = finished;
        });
    }else if ( !canLink ){
        showToast('Cannot link records: - Currently undergoing grading records');
    }else if ( !canGrade ){
        showToast('Cannot link records: - Currently undergoing linking records');
    }
});//END linkingRecords

Service

function linkRecords(batchSize){
    return $http.get(urlRoot+'/link/:'+batchSize);
}//END linkRecords

Node js api:

routes.js:

//link all records
app.post('/link/:batchSize', function(req, res){
    var batchSize = req.params.batchSize;
    functions.linkRecords(batchSize, function(callback){
        res.end(true);
    });
});//END linkRecords

functions.js:

linkRecords : function(batchSize, callback){
    //link all records in all collections a batchsize at a time
    async.parallel([
        function(callback){
            ...open cursor for collection1 and loops over the collections to stitch with other collections.
        },
        function(callback){
            ...open cursor for collection2 and loops over the collections to stitch with other collections.
        }
    ], function done(err, result){
        callback();
    });
}
3 Answers
Related