How do I remove the last character of a string if it's a backslash \ with pyspark?
I found this answer with python but I don't know how to apply it to pyspark:
my_string = my_string.rstrip('\\')
Remove last character if it's a backslash with pyspark
Viewed 441
1 Answers
You can use trim function:
import pyspark.sql.functions as F
df = spark.createDataFrame([("test\\",)], ["value"])
df.withColumn("value2", F.expr(r"trim(TRAILING '\\' FROM value)")).show()
#+-----+------+
#|value|value2|
#+-----+------+
#|test\| test|
#+-----+------+
Or using regexp_replace function:
df.withColumn("value2", F.regexp_replace("value", r"\\+$", "")).show()
Related