How to view Dask dashboard when running on a virtual machine?

user_1964692 0
Viewed 4155

Here is what I'm doing now:

  • From my Windows laptop, I SSH into Linux server via Putty: IP address is 11.11.11.111

  • Start up Jupyter notebook: nohup jupyter notebook --ip=0.0.0.0 --no-browser &

    • Terminal output shows Jupyter notebook is running at:
    • http://(11.11.11.111 or 127.0.0.1):8889/?token=blahblahblah

  • Start single machine client in a new notebook:

    from dask.distributed import Client
client = Client()

Printing client shows that the dashboard is located at http://127.0.0.1:8787/status, however, I cannot find the dashboard at that URL. I've also tried http://11.11.11.111:8787/status but that didn't work either.

I'm still able to run everything in my notebook with Dask Dataframes just fine, but I just can't figure out how to view the dashboard. Bokeh is installed on server and I'm running Jupyter Notebook through Anaconda.

dask centos virtual-machine python-3.x linux
1 Answers

Finally figured it out with some SSH Tunneling.

More background on problem:

  • Local machine is a Windows laptop
  • Remote server is a CentOS box

The goal is actually two-fold:

  1. Run Jupyter Notebook on remote server that contains Dask code

  2. View Dask Dashboard from code running in Notebook

Here are the steps I took:

  1. For this example, IP Address of remote server is 11.11.11.111

  2. Following some instructions for Port Tunneling, I use 8001 as the Source Port and Destination is localhost:8889

  3. After connecting to the remote server (which has 16 cores and 44.7GB of RAM), I ran this in Putty terminal: dask-worker tcp://11.11.11.111:8786 --memory-limit=auto --nthreads=1 --nprocs=16 &

  4. Start Jupyter Notebook on server: jupyter notebook --ip=0.0.0.0 --port=8889 --no-browser &

    a. After running above command, output shows that Jupyter notebook is running at http://(hostname or 127.0.0.1):8889/?token=blahblahblah

    b. Opening a browser and going to the URL above (http://hostname:8889/?token=blahblahblah) brings to Jupyter Notebook home page

  5. Create new Notebook and run following code:

    import dask.dataframe as dd
from dask.distributed import Client
client = Client('11.11.11.111:8786')
print(client)

The output shows dashboard 

    Client
    Scheduler: tcp://11.11.11.111:8786
    Dashboard: http://11.11.11.111:36124/status
    client = Client('11.11.11.111:8786')

    Cluster
    Workers: 16
    Cores: 16
    Memory: 44.70 GB

Now typing http://11.11.11.111:36124/status into a browser window takes me to the Dask Dashboard.

user_19646920
Related