I am evaluating dotnet Spark and in order to debug application, I have to copy the dlls to /usr/local/bin in the Databricks cluster.
Today, I am able to create a init script that copies the dlls from dbfs to /usr/local/bin while starting the cluster. Then, I start spark using databricks-connect and debug my dotnet application (as if spark is running in local).
I would like to automate this on a running cluster. That is, when I start debugging my dotnet application in Visual studio, it deletes the dlls in /usr/local/bin and then copy dlls from my local dotnet debug folder to /usr/local/bin.