Hive Where clause with subquery - Only SubQuery expressions that are top level conjuncts are allowed

user_7675381 0
Viewed 119

I need to get all the record from tableA greater than cdc_date which is stored in another tmp_table

tmp_table has only one column cdc_date and only one record.

tableA has more the 5 million records.

My Hive query

Select count(*) from tableA as a 
where unix_timestamp((concat_ws('-',a.year,a.month,a.day,a.hour)),"yyyy-MM-dd-HH") > 
(select b.cdc_date from tmp_table as b)

I am receiving below error

Unsupported SubQuery Expression 'cdc_date': Only SubQuery expressions that are top level conjuncts are allowed

Can anyone suggest how to active this.

non-equi-join subquery hive where-clause
2 Answers

You need to rewrite the sql -

Select count(*) from tableA as a 
Left join tmp_table b
On unix_timestamp((concat_ws('-',a.year,a.month,a.day,a.hour)),"yyyy-MM-dd-HH") > 
b.cdc_date
user_64305230

Cross join with single row table:

Select count(*) 
 from tableA as a 
      cross join tmp_table b
where unix_timestamp((concat_ws('-',a.year,a.month,a.day,a.hour)),"yyyy-MM-dd-HH") > b.cdc_date
user_27003440
Related