Alternative to Hive Query with NOT IN clause

user_106521 0
Viewed 1245

I have the following set of hive tables :

create table image_additions (
    customer_id STRING,
    image_key STRING,
    image_size STRING
);
create table image_removals (
   customer_id STRING,
   image_key STRING,
   image_size STRING
);
create table images_stored (
   customer_id STRING,
   image_key STRING,
   image_size STRING
);

And I want to run an insert from a query like the following :

insert into images_stored
select ia.customer_id, 
       ia.image_key, 
       ia.image_size 
from image_additions ia 
where ia.image_key not in 
       (select ir.image_key from image_removals ir);

This produces a cartesian product, and hive does not allow me to run it. How can I do this, using an alternative query ?

hiveql notin hive sql
1 Answers

Using left join + where is null;

insert into images_stored
select ia.customer_id, 
       ia.image_key, 
       ia.image_size 
from image_additions ia 
     left join image_removals ir on ia.image_key=ir.image_key 
where ir.image_key is null;

Using not exists:

insert into images_stored
select ia.customer_id, 
       ia.image_key, 
       ia.image_size 
from image_additions ia  
where not exists (select 1 from image_removals ir where ia.image_key=ir.image_key);
user_27003440
Related