Scala Future `.onComplete` function discarded after call?

user_42059 0
Viewed 140

Are the function bodies passed to Future.onComplete(), and their closures, discarded and so garbage collected after they are called?

I ask because I'm writing an unbounded sequence of Future instances. Each Future has an .onComplete { case Failure(t)...} that refers to the previous known-good value from a previous Future. What I want to avoid is the total history of all Future results being kept in the JVM's memory because of references in closure bodies.

Perhaps Scala is more clever than this, but skimming the code related to execution contexts and futures isn't yielding much.

Thanks.

concurrent.futures scala
1 Answers

The class that normally implements Future and that you want to look at is DefaultPromise.

It contains mutable state that is being updated as the Future completes.

  • If you call onComplete and it has already been completed then it just schedules your callback immediately with the result. The callback is not recorded anywhere.
  • If you call onComplete while the result is not yet available, the callback is added to a list of "listeners".
  • When the result becomes available (someone calls complete on the promise), then all listeners are scheduled to run with that result, and the list of listeners is deleted (the internal state changes to "completed with this result")

This means that your callback chain is only being built up until the "upstream future" is incomplete. After that, everything gets resolved and garbage-collected.

"list of listeners" above is a bit of a simplification. There is special care being taken that these listeners do not end up referring to each-other, specifically to break reference loops that would prevent garbage collection to work when constructing futures recursively. Apparently this was indeed a problem in earlier versions.

The problem of leaks is solved by automatically breaking these chains of promises, so that promises don't refer to each other in a long chain. This allows each promise to be individually collected. The idea is to "flatten" the chain of promises, so that instead of each promise pointing to its neighbour, they instead point directly the promise at the root of the chain. This means that only the root promise is referenced, and all the other promises are available for garbage collection as soon as they're no longer referenced by user code.

user_149550
Related