arithmetic operations with lenght function

BryC 0
Viewed 28

I have this Dataframe

+----+----+------+------+------+----+
|key1|key2|  col1|  col2|  col3|col4|
+----+----+------+------+------+----+
|   5|   d|value7|value8|value9|  20|
+----+----+------+------+------+----+

And I am trying to do something like this

df2.withColumn("new",repeat(lit("0"), 10-length(col("col3")) )).show()

But I get this error message TypeError: Column is not iterable

I would like to know if there is any way to do a subtraction or maybe an addition using "length("col3")"

databricks pyspark
1 Answers

Using repeat as SQL function instead of using the Python function works:

from pyspark.sql import functions as F

df.withColumn('new', F.expr('repeat("0", 10-length(col3))')).show()

Output:

+------+-----+
|  col3|  new|
+------+-----+
| hello|00000|
|value9| 0000|
+------+-----+
userf5W80
Related