I want to do some preprocessing on my data and I want to drop the rows that are sparse (for some threshold value).
For example I have a dataframe table with 10 features, and I have a row with 8 null value, then I want to drop it.
I found some related topics but I cannot find any useful information for my purpose.
stackoverflow.com/questions/3473778/count-number-of-nulls-in-a-row
Examples like in the link above won't work for me, because I want to do this preprocessing automatically. I cannot write the column names and do something accordingly.
So is there anyway to do this delete operation without using the column names in Apache Spark with scala?