Deprecated interfaces for implementing generic UDAF

Viewed 164

I want to implement my own custom UDAF. https://hive.apache.org/javadocs/r3.1.2/api/org/apache/hadoop/hive/ql/exec/UDAF.html This doc says:

Deprecated. 
Either implement GenericUDAFResolver2 or extend AbstractGenericUDAFResolver instead.

But both of these are deprecated as well, and there is no replacement specified. https://hive.apache.org/javadocs/r3.1.2/api/org/apache/hadoop/hive/ql/udf/generic/AbstractGenericUDAFResolver.html

What interface should I use for implementation and what am I missing here?

1 Answers

OK so I dug into it:

UDAFResolver2 Marked as deprecated and added GenericUDAFResolver2 & AbstractGenericUDAFResolver HIVE-287. Support count(*) and count distinct on multiple columns … John Sichi committed on Jul 11, 2010

Marked as deprecated --> GenericUDAFResolver2 & AbstractGenericUDAFResolver HIVE-17132: Add InterfaceAudience and InterfaceStability annotations … …for UDF APIs (Sahil Takiar, reviewed by Ashutosh Chauhan) master rel/storage-release-2.8.1 … rel/release-3.0.0 @sahilTakiar sahilTakiar committed on Aug 22, 2017

From the notes (HIVE-17132) it's not actually clear that UDAFs where intentionally added and included. In fact I might suggest that there was a well intended mistake when UDAFs were added and Resolver2 was depreciated. It seems this change wasn't really reviewed in depth. (I really can't be sure that there was intent to add UDAFS to this feature as there is no language that speaks to it. It seems to be accidentally lumped in with UDFs and this is where I think there was a mistake. I think it might have slipped through the cracks.)

Definitely there is a bug that it's not well documented what should replace resolver2. Please take a minute to read through the links and form your own opinion. It is worth sending an email to the dev list for hive and seeing what you get back from it.

I would continue to use AbstractGenericUDAFResolver.

Related