Many small requests vs few large requests - Angular to Django REST API - No DB involved

user_2542194 0
Viewed 459

I know there are related questions on SO, but I am not sure if the conditions I am asked to work on causes any changes to the answer, so I am asking it here.

I am creating a simple webapp in Angular that imports spreadsheet data from the user and sends the data to Django backend that does data analysis on it. The data results are returned to frontend and Angular creates a dashboard of the results. There is a chart displayed for every column of the spreadsheet. I am faced with two options:

a) Keeping the spreadsheet in the browser memory and sending each column of data separately to the Django server that makes analysis of the data and returns the results.

Pros: Simple architecture. No caching required.

Cons: If there are 150 columns in the sheet, it will result in 150 calls to the API for that user.

b) Sending the entire sheet of data and let python handle everything. It will return a big chunk of data in return which will have to be unpacked by Angular.

Pros: Only one request per file.

Cons: For subsequent calls for the same file, I might need caching? If the file has changed, it might cause stale data. I might also need to maintain sessions per user.

Restrictions I am working with: I cannot store the document either on the Django server or on the DB. Even though this is just an internal app, the docs could be sensitive and the users wont be comfortable of storage of any kind.

Also, there is a high probability that the files could be 100+ MBs in size, so that becomes a factor as well.

In such a scenario, does a "Many small requests" make more sense to implement? Apologies in advance if the question is duplicate.

angular backend client-server django python
1 Answers

Another factor which you are not considering is that browsers (lets say Chrome) only allocates 6 TCP ports per host. This means that if you go with the "Many small requests" approach, you may face some serious performance issues depending on how long it takes for that request to be processed on the backend.

Another factor of consideration is how are you going to handle data rollback? If you get through 50% (75) of the requests and the user wants to cancel or the browser crashes etc then what happens to the other 50%?

If this is an internal application running on a fast network I personally would just go with the bulk one request per file. 100MB on a 100GB network isn't that much of a drag.

If it is not on an internal network then I would have to choose the micro transactions, because the user experience of getting to 99% of an upload and failing after 10 mins is something that I am sure we have all experienced (awful). At least with the micro transaction approach you can control data rollback and even open up sockets on the front end to give updates. Like if a certain column failed during processing then angular can try and resend it etc.

There is no one size fits all here and this is merely my opinion.

user_35095910
Related