How to delete decimals and place zeros in front of number dynamically in pyspark?

user_15153279 0
Viewed 62

I have a pyspark dataframe and I want to delete everything after the . and I want to delete the . furthermore, I want to place zeroes in front of the numbers so every number has a length of 4.

As is:

|270.0            |  
|749.5            |  
|460.0            |  
|90.0             |  
|670.0            |    
|656.6666666666666|  
|1210.0           |

TO BE:

|0270|  
|0749|  
|0460|  
|0090|  
|0670|  
|0656|  
|1210|

Kind regards,

Wokter

pyspark apache-spark-sql apache-spark dataframe
1 Answers

You can use format_number + lpad functions:

from pyspark.sql import functions as F

df1 = df.withColumn(
    "number",
    F.expr("lpad(format_number(number, '####'), 4, '0')")
)

df1.show()
#+------+
#|number|
#+------+
#|  0270|
#|  0750|
#|  0460|
#|  0090|
#|  0670|
#|  0657|
#|  1210|
#+------+

Or lpad + regexp_replace :

df1 = df.withColumn(
    "number",
    F.lpad(F.regexp_replace("number", "\\.(.*)", ""), 4, '0')
)
user_13865510
Related