What happens when one query inserts and one alters the same table concurrently?

Viewed 138

What will happen if one or more processes are trying to append data to the table and one or more processes are trying to alter the table at the same time?

Let's say we have a table called db.table(partitioned on band column):

band |name |age
---------------------------
 1        abc   18
 1        xyz   20
 2        aaa   18
 2        bbb   19
 3        ccc   22
 4        ddd   28

Alter command:

spark.sql("alter table db.table add columns (year_of_birth int, salary double, city string)")

Data append:

df = spark.sql("select 5 as band, 'fff' as name, 33 as age")

df.write.partitionBy('band').mode('append').saveAsTable('db.table')

Here df.write and spark.sql(alter command) are executing at the same time by multiple processes. What is the behaviour of spark in this scenario?

1 Answers

tl;dr Easy, nothing surprising would really happen since the data to be written to would be loaded before that data (as a table) would be appended to.

Moreover, append queries are safe since they get data added (not really modified on row basis).

Related