Spark sql Resolved exception: attribute(s) missing from in operator !Project

Viewed 231

I am migrating some ETL sql from mysql to Spark sql . Encounter very strange exception . I hope some one can help me out. the test table :

create table spark_hdw.test_bug(
a decimal(20,2),
b decimal(20,2),
c decimal(21,2)
)

Spark sql:

%spark.sql
 
SELECT
    * 
FROM
    (
 
    SELECT a   AS kpi_04 
         
    FROM
        spark_hdw.test_bug 
    UNION ALL
    
    SELECT
        a+b     AS kpi_04 
    FROM
        spark_hdw.test_bug 
  
)  t

Exception:

Resolved attribute(s) kpi_04#17864 missing from kpi_04#17872 in operator !Project [kpi_04#17864]. Attribute(s) with the same name appear in the operation: kpi_04. Please check if the right attribute(s) are used.;;
!Project [kpi_04#17864]
+- SubqueryAlias t
   +- Union
      :- Project [cast(kpi_04#17864 as decimal(21,2)) AS kpi_04#17872]
      :  +- Project [a#17866 AS kpi_04#17864]
      :     +- SubqueryAlias test_bug
      :        +- HiveTableRelation `spark_hdw`.`test_bug`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, [a#17866, b#17867, c#17868]
      +- Project [kpi_04#17865]
         +- Project [CheckOverflow((promote_precision(cast(a#17869 as decimal(21,2))) + promote_precision(cast(b#17870 as decimal(21,2)))), DecimalType(21,2)) AS kpi_04#17865]
            +- SubqueryAlias test_bug
               +- HiveTableRelation `spark_hdw`.`test_bug`, org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe, [a#17869, b#17870, c#17871]
0 Answers
Related