How can I use databricks utils functions in PyCharm? I can't find appropriate pip package

user_10924136 0
Viewed 2550

PyCharm IDE. I want to use dbutils.widgets.get() in a module and than to import this module to databricks. I already tried with pip install databricks-client pip install databricks-utils and pip install DBUtils

pyspark databricks apache-spark pycharm python
1 Answers

The dbutils is available only as a part of the databricks-connect package. Its documentation contains detailed description on how to setup PyCharm to work with it. Ut also covers on how to use the dbutils.

You may need to define following wrapper to be able to use dbutils locally and on Databricks:

def get_dbutils(spark):
  from pyspark.dbutils import DBUtils
  return DBUtils(spark)

get_dbutils().fs.cp('file:/home/user/data.csv', 'dbfs:/uploads')
Alex_Ott_186270
Related