How to get the last row from DataFrame?

user_6676962 0
Viewed 30804

I hava a DataFrame,the DataFrame hava two column 'value' and 'timestamp',,the 'timestmp' is ordered,I want to get the last row of the DataFrame,what should I do?

this is my input:

+-----+---------+
|value|timestamp|
+-----+---------+
|    1|        1|
|    4|        2|
|    3|        3|
|    2|        4|
|    5|        5|
|    7|        6|
|    3|        7|
|    5|        8|
|    4|        9|
|   18|       10|
+-----+---------+

this is my code:

    val arr = Array((1,1),(4,2),(3,3),(2,4),(5,5),(7,6),(3,7),(5,8),(4,9),(18,10))
    var df=m_sparkCtx.parallelize(arr).toDF("value","timestamp")

this is my expected result:

+-----+---------+
|value|timestamp|
+-----+---------+
|   18|       10|
+-----+---------+
apache-spark-sql apache-spark scala
7 Answers

Java:

Dataset<Row> sortDF = inputDF.orderBy(org.apache.spark.sql.functions.col(config.getIncrementingColumn()).desc());
Row row = sortDF.first()
user_46475150

You can also use this function desc: Column desc(String columnName)

df.orderBy(desc("value")).show(1)

which gives same result as 

df.orderBy($"value".desc).show(1)
user_14655530
Related