I am writing a Spark Structured Streaming job and I have to filter on a string column.
When I am filtering on the long column it is able to filter(using below code):
arr = arr.filter(col("epoch") > 1610913720)
But when I am doing on a string, data is not filtered:
arr = arr.filter(col("id").equalTo("z12"))