How to split a number and add hyphen in a pyspark dataframe?

user_15153279 0
Viewed 921

I want to split all numbers in a column.

number: 123456789012

to be 123-4567890-12

So I want to add - at positions 4 and 10.

I do not want to make new columns

pyspark apache-spark-sql apache-spark dataframe split
2 Answers

You can use regexp_replace :

from pyspark.sql import functions as F

df1 = df.withColumn(
    "number",
    F.regexp_replace(F.col("number"), "(\\d{3})(\\d{7})(\\d+)", "$1-$2-$3")
)

df1.show()

#+--------------+
#|        number|
#+--------------+
#|123-4567890-12|
#+--------------+
user_13865510

You can slice the string using substring and combine them with a separator '-' using concat_ws:

import pyspark.sql.functions as F

df2 = df.withColumn(
    'number', 
    F.expr("""
        concat_ws(
            '-',
            substring(number, 1, 3),
            substring(number, 4, 7),
            substring(number, 11, length(number))
       )
    """)
)

df2.show()
+--------------+
|        number|
+--------------+
|123-4567890-12|
+--------------+
user_141657300
Related