Invalid literal number by running head command in azure databrick

user_2063547 0
Viewed 294

I want too run below command in my notebook. My language is Scala 2.11 and my spark version is 2.4.4

%fs head mnt/training/UbiqLog4UCI/14_F/log_1-6-2014.txt

I have below error:

<console>:6: error: Invalid literal number
%fs head /mnt/training/UbiqLog4UCI/14_F/log_1-6-2014.txt
                                   ^

Update for first comment

I add " and i have below error

enter image description here

databricks apache-spark scala
3 Answers

The issue is that in Scala 2.11, underscore is not accepted as a valid literal separator and hence, when it was parsing the path, it found undersore for as a separator and hence the issue. The solution to solve this could be as shown below: 

val cmd = "%fs head /mnt/training/UbiqLog4UCI/14_F/log_1-6-2014.txt"
import sys.process._
val exitcode = cmd.!

Please let me know if it helps!!

user_80628940

I found this workaround

val zipsDF = spark.read.json("/mnt/training/UbiqLog4UCI/14_F/log_1-6-2014.txt")
zipsDF.printSchema

enter image description here

user_20635470

You can get around with this issue by using dbutils.fs.ls command as below:

dbutils.fs.head("/mnt/training/UbiqLog4UCI/14_F/log_1-6-2014.txt")

user_48788480
Related