Apache Spark DataSet API : head(n:Int) vs take(n:Int)

user_4244240 0
Viewed 9246

Apache Spark Dataset API has two methods i.e, head(n:Int) and take(n:Int).

Dataset.Scala source contains

def take(n: Int): Array[T] = head(n)

Couldn't find any difference in execution code between these two functions. why do API has two different methods to yield the same result?

spark-dataframe apache-spark-sql apache-spark
4 Answers

I have experimented & found that head(n) and take(n) gives exactly same replica output. Both produces output in the form of ROW object only.

DF.head(2)

[Row(Transaction_date=u'1/2/2009 6:17', Product=u'Product1', Price=u'1200', Payment_Type=u'Mastercard', Name=u'carolina', City=u'Basildon', State=u'England', Country=u'United Kingdom'), Row(Transaction_date=u'1/2/2009 4:53', Product=u'Product2', Price=u'1200', Payment_Type=u'Visa', Name=u'Betina', City=u'Parkville', State=u'MO', Country=u'United States')]

DF.take(2)

[Row(Transaction_date=u'1/2/2009 6:17', Product=u'Product1', Price=u'1200', Payment_Type=u'Mastercard', Name=u'carolina', City=u'Basildon', State=u'England', Country=u'United Kingdom'), Row(Transaction_date=u'1/2/2009 4:53', Product=u'Product2', Price=u'1200', Payment_Type=u'Visa', Name=u'Betina', City=u'Parkville', State=u'MO', Country=u'United States')]

user_72622300 
  package org.apache.spark.sql
  /* ... */

  def take(n: Int): Array[T] = head(n)
user_95960530
Related