I have dataframe with one column containing base64 strings representing images, taken from jpg files read from Blob Container. Here simple code for getting it:
img_df = spark.read.format("binaryFile").load("/mnt/pictures")
proc_df = img_df.select(base64(col("content")).alias('encoded'))
proc_df.write.mode("overwrite").format("text").save('/mnt/pictures_text')
Now I want to save it as a image/binaryFile by loading text and decoding:
img_df = spark.read.format("text").load("/mnt/pictures_text")
proc_df = img_df.select(unbase64(col("value")).alias('decoded'))
proc_df.write.format("binaryFile").save("/mnt/pictures_text/copy")
It works fine until saving. I got exception that: Write is not supported for binary file data source. I also tried other ways, including e.g image mode. Is there any other way for saving it with jpg format? I can have images only as a base64 string in starting point.