I'm currently trying to write data using Iceberg to an external Hive table which is partitioned by partition_date column.
Before writing the data with Iceberg format, test table has 2 rows,
("2015-01-02", "S01233", "3-goods-purchased")
("2015-01-02", "S01234", "4-goods-purchased")
After writing data as below Code:
val input = Seq(("2015-01-02", "S01233", "5-goods-purchased"))
.toDF("partition_date", "order_id", "goods_purchased")
input.write
.format("iceberg")
.partitionBy("partition_date")
.option("path","s3://some-bucket-path/test")
.option(
"replaceWhere",
s"order_id in ('S01233')")
.mode("overwrite")
.saveAsTable("default.test")
Table test gets overwritten and only one row is shown in the output.
("2015-01-02", "S01233", "5-goods-purchased")
What I expected is this,
("2015-01-02", "S01233", "5-goods-purchased")
("2015-01-02", "S01234", "4-goods-purchased")
replaceWhere option provided as part of write statement, doesn't seem to work as expected.
Is there anything I'm missing? or does Iceberg support replaceWhere option which is working with delta format.?