OK, bunch of informations:
Scenario
- There is no UI.
- I need to do a lot of calculations on a server.
- As for now, the predicted time to finish them is a month.
- I need to parallelize the code, using the best techniques available. I can sacrifice a lot of time improving this.
- The synchronous part of the code is assumed to be written in an optimal way. The only possible way to improve the performance is to parallelize independent operations.
Nature of the calculations
- Operations I need to perform form a flow graph. And edge represents an fully independent operation. In vertex
AI start with only one task to do. When a task is being performed, it creates a bunch of other tasks to do. So in the very end I will have millions of tasks. Visualization:
- The vast majority of operations is really fast. They take like 100 milliseconds. Some of them, though, are longer. These are requests to external services.
A simple approach to just run all the operations asynchronously kills the machine - the overhead of creating millions of tasks is massive.
Question
How should I approach this problem? Parallel? PLINQ? Rx? Dataflow? Something else? Thread pool directly?