What is the pyspark equivalent of MERGE INTO for databricks delta lake?

user_1317167 0
Viewed 7875

The databricks documentation describes how to do a merge for delta-tables.

In SQL the syntax

MERGE INTO [db_name.]target_table [AS target_alias]
USING [db_name.]source_table [<time_travel_version>] [AS source_alias]
ON <merge_condition>
[ WHEN MATCHED [ AND <condition> ] THEN <matched_action> ]
[ WHEN MATCHED [ AND <condition> ] THEN <matched_action> ]
[ WHEN NOT MATCHED [ AND <condition> ]  THEN <not_matched_action> ]

can be used. Is a python-equivalent available?

delta-lake databricks delta merge
1 Answers

I managed to find the documentation using the help of Alexandros Biratsis. The documentation can be found here. An example of such a merge is given by

deltaTable.alias("events").merge(
    source = updatesDF.alias("updates"),
    condition = "events.eventId = updates.eventId"
  ).whenMatchedUpdate(set =
    {
      "data": "updates.data",
      "count": "events.count + 1"
    }
  ).whenNotMatchedInsert(values =
    {
      "date": "updates.date",
      "eventId": "updates.eventId",
      "data": "updates.data",
      "count": "1"
    }
  ).execute()
user_13171670
Related