I am trying to save a trained pyspark word2vec in local, however this is resulting in an error - Mkdirs failed to create
model.write().overwrite().save("word2vec.model")
22/08/31 12:56:24 WARN TaskSetManager: Lost task 0.0 in stage 421.0 (TID 11440) (100.66.40.74 executor 98): java.io.IOException: Mkdirs failed to create file:/home/jovyan/_git/notebooks/word2vec.model/metadata/_temporary/0/_temporary/attempt_202208311256242380698786646139780_0477_m_000000_0 (exists=false, cwd=file:/opt/spark/work-dir) at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:458)
The command does create a folder word2vec.model, but eventually fails. What can be the possible issue here?