How do I detect whether the SparkContext has been stopped?
How do I detect whether the SparkContext has been stopped?
If you use spark 1.5, that could be done via reflection API:
boolean isStopped(SparkContext sc) throws NoSuchFieldException, IllegalAccessException {
Field f = sc.getClass().getDeclaredField("stopped");
f.setAccessible(true);
AtomicBoolean stopped = (AtomicBoolean) f.get(sc);
return stopped.get();
}