Jupyter

Jupyter is a very useful web browser IDE for Python that is base on notebook format, similar to Mathematica or R markdown. Jupyter can be spawned on the compute node and the webpage can be accessed from the user through the SSH tunnel. JupyterLab provides a Jupyter Notebook environment, including the Python package.

There are three ways to get it running:

1. Use OnDemand Web Portal : https://ondemand-pioneer.case.edu to either use the "Jupyter Notebook" app or open a Jupyter Notebook on a compute node using "Pioneer Desktop"

2. Use the X2Go or other X11 access to the cluster, then request an interactive node with X11 (refer to HPC Guide to Visual Access).
   On this compute node, start jupyter and subsequently the browser to access jupyter.

3. Use SSH tunnel to allow local browser Jupyter Notebook.
   Connect to the cluster using 2 terminals: 

   1. to request an interactive node and to start jupyter, 

   2. to establish the ssh tunnel to the compute node.
      Once both terminals are set, open the browser on your local computer using the open jupyter port.

OnDemand Web Portal

There are two options:

1. Choose "Jupyter Notebook" option from the "Interactive Apps" and proceed accordingly.

2. Choose "Pioneer Desktop" option.

   • Adjust the requirements for the job that would run the Notebook, including requesting a GPU if that is needed.

   • Open the compute node desktop

   • Open a Terminal in the compute node desktop (right-click, hold, and choose Terminal in the menu)

   • Run "module load JupyterLab"

   • Run "jupyter-notebook --ip=127.0.0.1 &"

   • A new browser should pop up that runs the Jupyter Notebook.

X2Go and Other Direct X11 Cluster Access

Using X2Go is very similar to using the OnDemand Pioneer Desktop, because GUI is provided through the environment and you can directly run the Notebook on the compute node.

• Login to the cluster i.e. ssh into pioneer.case.edu with X-Forwarding with -X flag.

• Request a compute node. You can select a specific partition (e.g. gpu) if necessary.
  srun --x11 -t 2:00:00 -c 2 --mem=8gb --pty bash

• You will be assigned a compute node. Now, load JupyterLab module.
  module load JupyterLab

• Start jupyter notebook.
  jupyter notebook --ip=127.0.0.1 &

Using SSH Tunnel to run the Notebook on Local Computer Browser

The benefits of using the SSH tunnel is that you can run the Notebook on your local desktop's browser. This may improve the performance of the Notebook because the graphics rendering is all done by the local browser. However, it is a little bit more complex and you need to keep a couple of terminals open while running the Notebook.

1. On one terminal window, run the jupyter notebook

   • Login to the cluster i.e. ssh <CaseID>@pioneer.case.edu

   • Request a compute node. You can select a specific partition if necessary. The following provides default setting, including 10 hour duration.
     srun --pty bash

   • You will be assigned a compute node. Now, Load JupyterLab module
     module load JupyterLab

   • Start Jupyter on the interactive node.
     jupyter notebook --no-browser --ip=$(hostname -i) --port=9999

   • At the end of the output, it would say :
     To access the notebook, open this file in a browser:
             file:///home/hxd58/.local/share/jupyter/runtime/nbserver-2103518-open.html
         Or copy and paste one of these URLs:
             http://192.168.210.29:9999/?token=9f2ad574bffcc2f00b8a7f6f6f770fc0bb2051d62591539d
         or http://127.0.0.1:9999/?token=9f2ad574bffcc2f00b8a7f6f6f770fc0bb2051d62591539d

   • Note that the token number in this case is: 9f2ad574bffcc2f00b8a7f6f6f770fc0bb2051d62591539d
     
     

2. On the other terminal window, run the SSH tunnel

   • ssh -N -L 9999:<ip-address>:9999 <caseID>@pioneer.case.edu

   • It will appear to hang, but that is fine. You have imported the Notebook port (9999) to your local desktop browser

3. Keep both terminals open, and then open a local browser on your desktop.
   On the browser web address to load, enter: http://localhost:9999 and enter the token number