I have 2 tables as follows.
Need to join these 2 tables to get below table
I am trying different joins but not getting expected results. Could you please help me to get the desired table.
Really appreciate your help.
Thanks...
I have 2 tables as follows.
Need to join these 2 tables to get below table
I am trying different joins but not getting expected results. Could you please help me to get the desired table.
Really appreciate your help.
Thanks...
I don't know scala but in pyspark you can do the following:
df1.join(df2, 'DATE', 'full').fillna(0)
Essentially you do a full join and fill all the NULLs with 0.
For Hive SQL I guess it would be something like
SELECT Date,
CASE WHEN (table1.RM IS NOT NULL) THEN table1.RM ELSE 0 END AS RM,
CASE WHEN (table2.KM IS NOT NULL) THEN table2.KM ELSE 0 END AS KM
FROM table1
FULL JOIN table2
ON table1.Date = table2.Date
Hope this solution can help you, (I am used SQL_Server syntax)
SELECT isnull(date1,date2) as Date3, ISNULL(RM, 0 ),ISNULL(KM, 0 )
FROM table1
FULL JOIN table2
ON table1.Date1 = table2.Date2
order by Date3;
[RESULT]:
[EDIT]: Live demo
create table Table1 (DATE1 date, RM int); INSERT INTO Table1 VALUES ('1/4/2020' , 1); INSERT INTO Table1 VALUES ('2/1/2020' , 4); INSERT INTO Table1 VALUES ('2/10/2020' , 4);
GO
3 rows affected
create table Table2 (DATE2 date, KM int); INSERT INTO Table2 VALUES ('2/2/2020' , 1); INSERT INTO Table2 VALUES ('2/10/2020' , 3); INSERT INTO Table2 VALUES ('3/5/2020' , 2); GO
3 rows affected
select * from Table1; GODATE1 | RM :--------- | -: 2020-01-04 | 1 2020-02-01 | 4 2020-02-10 | 4
select * from Table2; GODATE2 | KM :--------- | -: 2020-02-02 | 1 2020-02-10 | 3 2020-03-05 | 2
SELECT isnull(date1,date2) as Date3, ISNULL(RM, 0 ),ISNULL(KM, 0 ) FROM table1 FULL JOIN table2 ON table1.Date1 = table2.Date2 order by Date3; GODate3 | (No column name) | (No column name) :--------- | ---------------: | ---------------: 2020-01-04 | 1 | 0 2020-02-01 | 4 | 0 2020-02-02 | 0 | 1 2020-02-10 | 4 | 3 2020-03-05 | 0 | 2
db<>fiddle here
I have created two initial dataframe named as df_rm, df_km as a source for your data.
df_rm looks like this:
+---------+---+
| date| rm|
+---------+---+
| 1/4/2020| 1|
| 2/1/2020| 4|
|2/10/2020| 4|
+---------+---+
df_km:
+---------+---+
| date| km|
+---------+---+
| 2/2/2020| 1|
|2/10/2020| 3|
| 3/5/2020| 2|
+---------+---+
Now, first we can do outer join then replace the null values with some values, in this case 0.
df_km.join(right = df_rm, Seq("date"),joinType = "outer")
.withColumn("rm",when(col("rm").isNull,0).otherwise(col("rm")))
.withColumn("km",when(col("km").isNull,0).otherwise(col("km")))
.show()
Which outputs like this:
+---------+---+---+
| date| km| rm|
+---------+---+---+
| 3/5/2020| 2| 0|
| 2/2/2020| 1| 0|
| 2/1/2020| 0| 4|
| 1/4/2020| 0| 1|
|2/10/2020| 3| 4|
+---------+---+---+