Progress bar with bootstrap, ajax laravel

Question

I´m trying to create a progress bar while uploading files and importing data in database.

I have created an importer with laravel and ajax with jquery. In this importer I´m reading all data from a CSV file. I´m getting all rows and cols from this file and while I´m reading this file, I´m doing a insert query with eloquent. This is my code:

$fp = fopen($route, 'r');

        // header
        $head = fgetcsv($fp, 4096, ';', '"');
        $header = [];
        
        $header[0] = 'nomape';
        $header[1] = 'direccion';
        $header[2] = 'cp';
        $header[3] = 'ciudad';
        $header[4] = 'provincia';
        $header[5] = 'telefono';
        $header[6] = 'movil';

        $rows = 0;
        $cells = 0;
        
        // Rows
        while($column = fgetcsv($fp, 8192, ';', '"'))
        {
            // build array to can combine header and this array
            $columns = [];
            
            foreach ($column as $val) {
                array_push($columns, $this->delete_accent($val));
            }

            $column = array_combine($header, $columns);
            $result = Listado::insert($column);
            
            if($result){
                $rows++;
            }
            
            $cells++;
        }
        
        fclose($fp);
        return 'Importados un total de: ' .$rows. ' de un total de: ' . $cells;

These operations take time. In my web browser console in network tab, I can show my http petition in pending while importing all my data. My CSV has a 69000 register.

With this code I´m building my progress bar:

$.ajax({    
                    xhr: function() {
                        var xhr = new window.XMLHttpRequest();
                        xhr.upload.addEventListener("progress", function(evt) {
                            if (evt.lengthComputable) {
                                var percentComplete = (evt.loaded / evt.total) * 100;
                                // Place upload progress bar visibility code here
                                console.log(percentComplete);
                            }
                        }, false);
                        return xhr;
                    },

But this returns 100% value in seconds, when these operations take many minutes. My question is how can I get real progress to create my progress bar ?

user_4369919 · Answer

You need to change your approach and do the csv traitement in 2 steps while repeating the second steps till the end of file.

First step With ajax, do a normal upload without processing the data. the request should return the uploaded file identifier with the row counts.

Second step With ajax, call for the first 100 rows to be processed (change the 100 value to your liking)

Once the request responds, update the progress bar (since you have the total count of the rows (100/69000 alias 0.1%) and call for the next 100 entries to be processed.

repeat this till you reach the row count. At the last process, you can delete the file.

PS: you will need some way to clean up trash uploads (those wich their process has not reached 100%) You can do that by cleaverly naming the uploaded file. For example, add the date of upload (plus unique id) to the file name and delete anything older than 24h.

user_3807365 · Answer

For uploading you can use client side to notify of progress (if you're using ajax).

xhr.upload.addEventListener("progress", function(evt) {
    if (evt.lengthComputable) {
        var percentComplete = evt.loaded / evt.total  * 100 >>>0;
        // Do something with download progress
        console.log("upload" + percentComplete + "%")

    }
}, false);

For importing and server processing progress you can use A. WebSockets server or B. an API service provider (such as rejax.io) for this. Then whenever a progress is made in server you make a call to API and client(s) will receive.

// if using PHP for example.
Rejax::send($str_channel_name, $str_message);

Proper discloser: I run https://rejax.io service

Progress bar with bootstrap, ajax laravel

2 Answers