What should be more efficient - Appending results to a new table or Inner Join with the existing table

I have a Original_Table that contains an empty column called Results (which is to be populated using python code being run on other columns in Original_Table). The primary key in this Original_Table is ROWID

I also have an empty (currently) table called Results_Table that has the same columns/data format as Original_Table

What I am doing currently:

1. The Original_Table contains 1 million records , so I am creating batches of 100 samples as a Temporary_Small_Table and calculating Results column.
2. Then I append the Temporary_Small_Table to the initially empty Results_Table

As you can notice , I do not update the Original_Table with the data in my Temporary_Small_Table as I imagine a join across Temporary_Small_Table (100 records) X Original_Table(1 million records) will be very inefficient. So I simply keep on appending my Temporary_Small_Table to the Results_Table.

I wanted recommendations on what's the correct way to do this (Joining to Original_Table or Appending to Results_Table ).

One obvious consequence of Appending to Results_Table is wasting storage space for a brand new table.

Just few additional background points:

1. I am creating batches for running my python code in multiple threads.
2. I am doing this on the Azure Databricks platform. This link tells me the database is MySQL and PostgreSQL operated as a service by Azure and AWS.
3. The join I am speaking about will be done on ROWID(it's the primary key as mentioned in the beginning)
4. For point #1 , I am already splitting my batches with predetermined row ids in the table , hence different threads picking up the same row from the Original_Table is not an issue at the moment for me. How to avoid the same row from the Original_Table from getting picked up without predetermining row ids for each batch is a different topic which I would park for later