By navigating to the dask dashboard, I see these 3 items being displayed
What does 'released', 'memory' and 'processing' exactly mean in this context? When they change from the left to the right side, what is happening with my computations? Released means that the memory was released because the computation is done?
Dask tracks the status/state of every item in the task graph (DAG) and the possible states are:
processing: dask received a task and it is being processed by one of the workers, so this refers to active computation happening on a worker
waiting: task was received, but it depends on another task that is still being computed, there is no active computation happening for this task
memory: worker finished the computation and the result is held on the worker, typically this is a transitory state (so the memory will be cleared as soon as the object is not needed), but it is possible to maintain objects in memory indefinitely with client.persist
released: worker released the object from memory, this is usually because all the tasks that need this object received a copy of the object, so there is no longer need to maintain this object in memory (but it is also possible to trigger this manually with future.release())
erred: worker encountered an error when computing the task, this is a catch-all for syntax errors, memory errors, IO errors, etc. The details will be available in the worker log
There are two additional values: no-worker state and forgotten status, you can find more details in the official documentation.
