How split between Number pyspark or NLP

One way using SparkSQL's builtin functions sentences() and flatten() [need spark 2.4.0+ for flatten()]:

from pyspark.sql.functions import expr

df.withColumn('new_s', expr('flatten(sentences(s))')).show(truncate=False)    
#+---------------------------------------------+----------------------------------------------------+
#|s                                            |new_s                                               |
#+---------------------------------------------+----------------------------------------------------+
#|Cras mattis MP the -69661/69662;69663 /IS4567|[Cras, mattis, MP, the, 69661, 69662, 69663, IS4567]|
#+---------------------------------------------+----------------------------------------------------+

what sentences() does from Apache Hive documentation:

Tokenizes a string of natural language text into words and sentences, where each sentence is broken at the appropriate sentence boundary and returned as an array of words. The 'lang' and 'locale' are optional arguments. For example, sentences('Hello there! How are you?') returns ( ("Hello", "there"), ("How", "are", "you") ).

You can use split functions which takes regex pattern to split data.

import pyspark.sql.functions as f

df.withColumn('ns', f.split('s', "[^a-zA-Z0-9']+")).show(10,False)
+---------------------------------------------+----------------------------------------------------+
|s                                            |ns                                                  |
+---------------------------------------------+----------------------------------------------------+
|Cras mattis MP the -69661/69662;69663 /IS4567|[Cras, mattis, MP, the, 69661, 69662, 69663, IS4567]|
+---------------------------------------------+----------------------------------------------------+

Note : "[^a-zA-Z0-9']+" will take care of basic English characters But if you want to include special characters, you can use generic \p{L} instead of a-zA-Z Like "[^\\p{L}0-9']+"