How to write rows asynchronously in Spark Streaming application to speed up batch execution?

I have a spark job where I need to write the output of the SQL query every micro-batch. Write is a expensive operation perf wise and is causing the batch execution time to exceed the batch interval.

I am looking for ways to improve the performance of write.

1. Is doing the write action in a separate thread asynchronously like shown below a good option?

2. Would this cause any side effects because Spark itself executes in a distributed manner?

3. Are there other/better ways of speeding up the write?

   // Create a fixed thread pool to execute asynchronous tasks
   val executorService = Executors.newFixedThreadPool(2)
   dstream.foreachRDD { rdd =>
     import org.apache.spark.sql._
     val spark = SparkSession.builder.config(rdd.sparkContext.getConf).getOrCreate
     import spark.implicits._
     import spark.sql
   
     val records = rdd.toDF("record")
     records.createOrReplaceTempView("records")
     val result = spark.sql("select * from records")
   
     // Submit a asynchronous task to write
     executorService.submit {
       new Runnable {
         override def run(): Unit = {
           result.write.parquet(output)
         }
       }
     }
   }