This method is somewhat insecure (attackers within the Graham network can potentially send commands to your notebook server.) It should be re-written to follow the "Jupyter on a HPF qlogin node" section of the main Jupyter wiki page. I'll try to do this shortly – let me know if you need this.
Refer to Pydpiper on Graham for getting started with Compute Canada systems, running Pydpiper and MICe-lab software on a Compute Canada cluster, and other tips.
On the Graham cluster, Network Address Translation (NAT) is blocked by default at the firewall, preventing users from accessing the internet from the compute nodes. Although one can contact technical support to have a specific port opened, the user would have to supply the IP address or range of addresses which should be allowed to use that port. (Ref: https://docs.computecanada.ca/wiki/SSH_tunnelling#Connecting_to_a_program_running_on_a_compute_node)
Cedar has the exact same default software modules as Graham, and very similar hardware but without these firewall issues. You can run Jupyter notebooks on a compute node with a GPU.For a list of GPUs available and how to refer to them, see https://docs.computecanada.ca/wiki/Using_GPUs_with_Slurm.
nzxwang@cedar.computecanada.ca's password: |
Replace "nzxwang" with your username, and then enter your password. You should be on a Cedar login node.
$ salloc --time=24:00:00 --cpus-per-task=4 --mem-per-cpu 4G --account=def-jlerch
If you want a GPU:
$ salloc --time=24:00:00 --gres=gpu:1 --cpus-per-task=4 --mem-per-cpu 4G --account=def-jlerch
This gets an arbitrary GPU. If you want a specific type, consult the CC wiki page referenced above for the appropriate type specifier, e.g., for a V100 GPU with 32GB of RAM, use --gres=gpu:v100l:1.
Once on the node, run this script:
$ cat /home/nzxwang/venv/fastai/bin/notebook.sh #!/bin/bash $ bash /home/nzxwang/venv/fastai/bin/notebook.sh [I 09:54:49.044 NotebookApp] Serving notebooks from local directory: /scratch/nzxwang |
On your local machine, create an SSH tunnel to the Graham login nodes:
$ ssh -f -N -L localhost:8880:cdr255.int.cedar.computecanada.ca:8888 nzxwang@cedar.computecanada.ca nzxwang@cedar.computecanada.ca's password: |
Then open up localhost:8880 in the browser and enter in the token from the last part of the URL.