I have to process around 200.000 objects (in a desktop application) and each object takes around 20 ms to process. In order to speed this up I want to do it concurrently.
For testing I just put each object in a separate task, but due to the small size of the job this only yields a tiny speed improvement. So my first question is:
Is there a clever (but not too complicated) way to find a optimal batch size for these objects? I guess I could to some local testing on whether it is fastest to group them together in batches of 10, 20 or 100 objects, but this seems a bit suboptimal.
Secondly (and more important): Most of the objects should just be processed whenever they get some CPU time. However, the user will always be looking at 10-20 objects. I want to always be able to put the objects the user is looking on in the front of the queue in order to deliver a smooth user experience. The user might navigate around all the time so I feel it is important to always be able to quickly reschedule the order. (20 ms * 20 should be able to be processed in around 0.4 seconds).
Can someone help me with a good design patten for processing these objects?