How to skip unavailable columns when using VectorSlicer in pyspark?

Viewed 9

I am using a VectorSlicer like this:

features_slicer = VectorSlicer(
    inputCol="features",
    outputCol="important_features",
    names=important_features,
)

Here, I am selecting only those columns from features vector which are from important_features list. But sometimes important_features list might contain a column that's not available in features vector. How do I skip these columns?

Something like:

features_slicer = VectorSlicer(
    inputCol="features",
    outputCol="important_features",
    names=important_features,
    ignore_unavailble_columns=True,  # is something like this available?
)

0 Answers
Related