I'm in the middle of a SparkStreaming project and I find myself doing many writeStreams with query names. Attached is an example:
.format("memory") \
.queryName("infoGames") \
.outputMode("complete") \
.start()
It's beautiful and pretty but at the end it annoys me... Impossible to restart again because the queryname exists.
I would therefore like to be able to DELETE the queryname that I created in order to be able to writeStream again and avoid doing a Sparksession.stop()
Do you have an idea ?
Thanks in advance for your feedback !