Jupyter on a compute node

The following steps demo:

  1. Create a job submission script to run a Jupyter notebook on the Discovery cluster
  2. Submit the job to the scheduler
  3. Create a SSH tunnel to Discovery and browse to the notebook

Create a file named jupyter.sh with the following contents:

#!/bin/bash
#SBATCH --job-name=jupyter_notebook
#SBATCH --time=10:00:00
#SBATCH --output=jupyter_notebook_%j.txt
#SBATCH --error=jupyter_notebook_%j.err

# get tunneling info
XDG_RUNTIME_DIR=""
node=$(hostname -s)
user=$(whoami)
cluster="discovery"
port=8888

# print tunneling instructions jupyter-log
echo -e "
# Command to create SSH tunnel:
ssh -N -f -L ${port}:${node}:${port} ${user}@${cluster}.dartmouth.edu

# Use a browser on your local machine to go to:
http://localhost:${port}/
"

module load python
jupyter-notebook --no-browser --port=${port} --ip=${node}

# keep it alive 
sleep 36000

Next setup a password:

$ module load python
$ jupyter notebook --generate-config
$ jupyter notebook password

<Enter a password and confirmation password>

Once complete, submit the job:

$ sbatch jupyter.sh

When the job starts, take a look at the jupyter_notebook_<job_id>.txt file to see the SSH tunnel command needed

$ cat jupyter_notebook_<job_id>.txt
# Command to create ssh tunnel:
ssh -N -f -L 8888:j04:8888 netid@discovery.dartmouth.edu

Use a browser on your local machine to go to:
http://localhost:8888/

Details

Article ID: 76952
Created
Fri 4/26/19 3:30 PM
Modified
Fri 7/23/21 11:28 PM