So, I’m trying to figure out the behaviour of static variables in Beam DoFn, is it shared between threads (within the same JVM)?
Basically trying to understand the following from the programming guide:
4.3.2. Thread-compatibility
… Note that static members in your function object are not passed to worker instances and that multiple instances of your function may be accessed from different threads.
Now, it its seems that the following static object "counter" got initialized, serialized and applied in the worker (Flink engine), is it aligned with the above statement?
If worker threads falls in different processes/JVM’s, obviously will not be shared. But if falling to same JVM will "counter" be shared ?
public class myTransform extends DoFn<KV<String >,String> implements Serializable {
private static AtomicLong counter = new AtomicLong(0);
...
@ProcessElement
public void processElement(ProcessContext c) {
...
counter.incrementAndGet();
}
}
thanks