What does ":_*" mean in scala? (When using a List to filter a dataframe)

user_12645016 0
Viewed 1698

When seeing some co-workers' Scala-Spark code, sometimes I encounter that they use lists to filter dataframes as in this example:

val myList: List[String] = List("0661", "0239", "0949", "0380", "0279", "0311")

df.filter(col("col1").isin(myList:_*)

The code above works perfectly, this one, however, does not:

df.filter(col("col1").isin(myList)

What I don't understand is, what is that "colon underscore star" :_* exactly doing?

Thanks in advance!

apache-spark underscore.js dataframe list scala
3 Answers

It does mean "pass list as a separate parameters". It works for methods, that have a vararg argument, like "any number of strings", but not a List[String] version.

Spark's isin function has signature isin(list: Any*): Column, Any* means "any number of arguments of type Any". Not very descriptive, but here you can pass either any number of strings, or any number of cols. With :_* syntax, you're saying to compiler "replace my list with varargs", it's equialent to writing .isin("0661", "0239" ...)

Also, since Spark 2.4.0 there's function isInCollection, that takes Iterable, so you can pass List there directly.

user_27958570

This is sometimes called splat operator. It is used to to adapt a sequence (Array, List, Seq, Vector, etc.) so it can be passed as an argument for a varargs method parameter:

def printAll(strings: String*):Unit = {
    strings.foreach(println)
  }

val fruits = List("apple", "banana", "cherry")
printAll(fruits:_*)
user_25012790

If any method contains any repeated parameter. If you want to pass any Iterable in the method's repeated parameter, to convert your Iterable to repeated parameter you will use :_*

def x(y:Int*):Seq[Int]={  // y:Int* is a repeated parameter. 
 y
}


x(List(1,2,3,4):_*)  <--- you are passing List into repeated parameter
user_75853270
Related