I'm using azure databricks, and have no need for the R language extensions. However I'm finding lots of initialization output related to R in the driver logs.
The initialization of the cluster is taking a surprising amount of time, even after setting up instance pools. When I started investigating, I discovered that there are many things happening during initialization, some of which I don't care about. There are even R-related processes being started in the cluster:
21/03/26 15:46:38 INFO RDriverLocal: 5. RDriverLocal.e6356ff0-aab3-4d12-a0df-047a3e8f9753: launching R process ...
21/03/26 15:46:38 INFO RDriverLocal: 6. RDriverLocal.e6356ff0-aab3-4d12-a0df-047a3e8f9753: cgroup isolation disabled, not placing R process in REPL cgroup.
21/03/26 15:46:38 INFO RDriverLocal: 7. RDriverLocal.e6356ff0-aab3-4d12-a0df-047a3e8f9753: starting R process on port 1100 (attempt 1) ...
21/03/26 15:46:38 INFO RDriverLocal: 8. RDriverLocal.e6356ff0-aab3-4d12-a0df-047a3e8f9753: setting up BufferedStreamThread with bufferSize: 100.
21/03/26 15:46:40 INFO RDriverLocal: 9. RDriverLocal.e6356ff0-aab3-4d12-a0df-047a3e8f9753: R process started with RServe listening on port 1100.
21/03/26 15:46:41 INFO RDriverLocal: 10. RDriverLocal.e6356ff0-aab3-4d12-a0df-047a3e8f9753: starting interpreter to talk to R process ...
Can someone tell me if there is any way to disable all this R cruft? If that is possible then I'm guessing it will save ten seconds or so. It doesn't sound like much but when you are launching job clusters all day long from an ADF pipeline in Azure, it gets a bit discouraging to think about how much time is consumed performing useless initialization operations.