How to delete folder/files from Databricks mnt directory

user_6513537 0
Viewed 8421

I am running Databricks Community Edition and I would like to delete files from the following mnt directory

/mnt/driver-daemon/jars

I run the dbutils command:

dbutils.fs.rm('/mnt/driver-daemon/jars/', True)

However, when I run the command I get the following message (which basically means the folder hasn't been deleted)

Out[1]: False

Can someone let me know where I going wrong? Ideally, I would like to delete all the files in the jars folder, however, if someone could just help showing how to delete the folder, that will suffice.

azure-databricks databricks
2 Answers

If dbutils.fs.rm() does not work you can always use the the %fs FileSystem magic commands. To remove a director you can use the following.

%fs rm -r /mnt/driver-daemon/jars/

where

  • %fs magic command to use dbutils
  • rm remove command
  • -r recursive flag to delete a directory and all its contents
  • /mnt/driver-daemon/jars/ path to directory
user_84148550

If you are referring to local /mnt/driver-daemon/jars ,it wouldn't be good idea to remove it since it is symlink to /databricks/jars directory and would impact driver functionality. The path /mnt/driver-daemon/jars/ resolves to dbfs:/mnt/driver-daemon/jars/ whose equivalent local file system directory /dbfs/mnt/driver-daemon/jars. If you want to delete local file system directories, you can prefix file: schema before the path (like file:/tmp/deleteme) with dbutils commands.

 %sh ls -ltrh /dbfs/mnt/driver-daemon/jars /mnt/driver-daemon/jars 
      ls: cannot access '/dbfs/mnt/driver-daemon/jars': No such file or directory
      lrwxrwxrwx 1 root root 16 Jul  2 23:14 /mnt/driver-daemon/jars -> /databricks/jars
user_12745620
Related