Apache Flink: Is MapState automatically updated when I modify a stored object?

user_10422561 0
Viewed 593

Is it necessary to use MapState.put() to manually update the state or whether is the state automatically updated when I modify an object?

private transient MapState<String, Word> words;
.......
Word w = words.get(word);             
if (w == null) {
  w = new Word(word);
  //words.put(word, w);  //A
}              
if (....) {
  w.countBad(1);   // countXXX modifies a the private variable in a Word object 
} else {
  w.countGood(1);
}    
//words.put(word, w);   //B

Q: If I use the A method, will the next count calculation automatically update the corresponding Mapstate state? Or do I need to use the B method to manually update the state after the calculation is complete?

flink-streaming apache-flink
1 Answers

From an API point of view, you always need to manually update the state.

However, the actual behavior depends on the state backend. If the application uses the InMemoryStateBackend or the FsStateBackend, all local state is stored on the JVM heap of the worker process, i.e., the state backend just holds a reference to the object. Hence, the state is directly modified when you modify the object.

If you use the RocksDBStateBackend all state accesses are de/serialized and read from / written to RocksDB. In this case modifying the object does not have an effect on the state.

I recommend to always explicitly update the state because this will ensure that you can switch the state backend without adjusting the logic of your application.

user_36095710
Related