Intro to HPC Workshop

To access the slides and information on this workshop, see our Training Documentation.

vertical-hpc

Submitting a Batch Job

Accessing Workshop Files
Submission Script Details
Submitting Your Job
Job Output

Overview

This short training class is available to users periodically throughout each academic year. Information on upcoming workshops can be found in our training documentation. Before each workshop, an email will be sent to all HPC users through hpc-announce with information on dates, times, location, and how to register.

This training course will help you get started on using HPC resources. If you have recently started, you might learn there is a better way, or there are capabilities you are not taking advantage of. You may also want to go through our quick start for additional information.

A PDF of the slides from past workshops can be found in our training documentation linked above and can be downloaded by clicking the Download PDF button at the top of this page. A summary of the training material for the interactive component of this workshop can be found below.

System Basics

Logging In

cluster-diagram

Logging in can be done two ways, either using your own terminal/SSH client or through our Open OnDemand web interface. Both methods are described below.

More detailed information can be found in our online documentation.

Local terminal

Mac/Linux

On a Mac or Linux, open a Terminal (on Mac, this can be found under Applications -> Utilities), and connect to the system using the following as a guide:

💻 intro-to-hpc $ ssh your_netid@hpc.arizona.edu
Password: # <- your password goes here
Duo two-factor login for your_netid

Enter a passcode or select one of the following options:

 1. Duo Push to XXX-XXX-XXXX
 2. SMS passcodes to XXX-XXX-XXXX (next code starts with: X)

Passcode or option (1-2): 1
Success. Logging you in...
This is a bastion host used to access the rest of the RT/HPC environment.

Type "shell" to access the job submission hosts for all environments
-----------------------------------------

[your_netid@gatekeeper 16:26:46 ~]$ shell
***
The default cluster for job submission is Puma
***
Shortcut commands change the target cluster
-----------------------------------------
Puma:
$ puma
(puma) $
Ocelote:
$ ocelote
(ocelote) $
ElGato:
$ elgato
(elgato) $
-----------------------------------------

(puma) [your_netid@wentletrap ~]$ ocelote
(ocelote) [your_netid@wentletrap ~]$ 

Windows

putty-login

On Windows, you will need an SSH client such as PuTTY or MobaXterm. In PuTTY, use hpc.arizona.edu as your hostname, click Open, then enter your NetID at the login as promt. Once you have duo-authenticated, type shell, then ocelote to connect to Ocelote.

Open OnDemand

Log into https://ood.hpc.arizona.edu/, then select the Clusters dropdown tab and click Shell Access

ood_shell_access

Once you’re connected, type ocelote to connect to Ocelote.

Working on the Command Line

Once you’re logged in, you can interact with the system. Some commands are provided below.

Linux Shell

Command	What it does
`ls`	Lists the contents of a directory
`pwd`	Prints the location of your working directory
`whoami`	Prints your username
`hostname`	Prints the name of the machine you’re connected to
`which <command>`	Prints the location of the executable `<command>`
`mkdir <directory_name>`	Create a directory with name `<directory_name>`
`cd`	Change directories
`cp <source> <destination>`	Copies a file/directory from `source` to `destination`
`cat <filename>`	Prints the contents of a text file `<filename>` to the terminal

HPC-Specific

Command	What it does
`uquota`	Checks your storage usage
`va`	Checks your group’s monthly time allocation
`nodes-busy`	Displays a visual of a cluster’s compute nodes and their usage
`system-busy`	Displays a text-based summary of a cluster’s compute nodes and their usage
`cluster-busy`	Displays a visual of each cluster’s average usage. Only available on the login nodes
`job-history <jobid>`	Prints a detailed summary of a running/completed job
`seff <jobid>`	Prints a summary of a completed job’s memory and CPU efficiency

Transferring Files

Uploading small files can be done through Open Ondemand by going to the Files dropdown. Select the relevant directory location, e.g. your home directory, and click upload:

select-home

upload

Drag/drop your files, or click Browse Files to upload

drag-drop file

Once uploaded, your files will appear in the relevant location

test

For larger files, see our online documentation for better transfer strategies.

Submitting a Batch Job

Accessing Workshop Files

To access the files you need for this workshop, use the following commands:

mkdir intro-to-hpc
cd intro-to-hpc
wget https://ua-researchcomputing-hpc.github.io/Intro-to-HPC/intro-to-hpc.tar.gz
tar xzvf intro-to-hpc.tar.gz --strip-components=1
rm intro-to-hpc.tar.gz

Submission Script Overview

The bash command cat will print a file’s contents to your screen. View the contents of the sample script you just copied using:

cat sample_slurm.script

This should display the following:

#!/bin/bash
#SBATCH --job-name=test
#SBATCH -e test.e%A
#SBATCH -o test.o%A
#SBATCH --partition=windfall
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --time=00:10:00
echo 'This script is running on:'
hostname
sleep 120

Note that there are three components to the script above:

The first line #!/bin/bash is called a “shebang” and specifies the language interpreter which will be used to execute the script. In this case, it’s set to Bash.
Lines beginning with #SBATCH are SLURM directives used to set job specifications. This include requesting compute resources, setting output filenames, requesting email notifications, etc.
Everything after the last #SBATCH are the commands that will be executed on the compute node(s).

Submitting Your Batch Job

The command sbatch will submit your script to the job scheduler to run on one of Ocelote’s compute nodes:

sbatch sample_slurm.script

If everything worked correctly, a job ID will be printed to the screen that you can use to track it’s progress using:

squeue --job <your_job_id>

For example:

(ocelote) [netid@wentletrap intro-to-hpc]$ sbatch sample_slurm.script 
Submitted batch job 463482
(ocelote) [netid@wentletrap intro-to-hpc]$ squeue --job 463482
             JOBID PARTITION     NAME  USER ST       TIME  NODES NODELIST(REASON)
            463482  windfall     test netid PD       0:00      1 (Priority)

Job Output

Once your job starts running, an output file will be created in the directory where you submitted your script. In this case, you will have two output files that will look like test.o<jobid> and test.e<jobid>. The first will be any standard output that would have been printed to the terminal had you run your work interactively and the second will contain any errors. First, check if you have output files using ls. If they exist, check their contents using cat. For example:

(ocelote) [netid@wentletrap intro-to-hpc]$ ls
test.e463482  test.o463482  sample_slurm.script
(ocelote) [netid@wentletrap intro-to-hpc]$ cat test.o463482 
This script is running on:
i11n1.ocelote.hpc.arizona.edu

Interactive vs. Batch

Sometimes you need to work with your analyses interactively, for example:

To compile code
To execute test runs
To test software/system modules

The built-in command interactive will put you on a compute node where you’ll have access to compute resources and system modules. You’ll notice your command prompt will change its hostname once your session starts to reflect the machine you’re connected to. For example:

(ocelote) [netid@junonia ~]$ interactive
Run "interactive -h for help customizing interactive use"
Submitting with /usr/local/bin/salloc --job-name=interactive --mem-per-cpu=4GB --nodes=1    --ntasks=1 --time=01:00:00 --account=windfall --partition=windfall
salloc: Pending job allocation 464177
salloc: job 464177 queued and waiting for resources
salloc: job 464177 has been allocated resources
salloc: Granted job allocation 464177
salloc: Waiting for resource configuration
salloc: Nodes i15n0 are ready for job
[netid@i15n0 ~]$ 

Try running interactive --help for information on full usage.

Accessing Software

Software packages are installed on HPC’s compute nodes as software modules. Note that software modules are not available on the login nodes. See the section on Interactive Sessions for accessing a compute node.

Modules on the Command Line

Software modules are accessible using the command module:

Command	Description	Example
`module avail`	Prints available modules on the system	`module avail python`
`module load`	Adds software to your environment	`module load python/3.6`
`module unload`	Unloads software from your environment	`module unload python/3.6`
`module swap`	Swaps software versions in your environment	`module swap python/3.6 python/3.8`

Software Through Open OnDemand

Some software packages can also be used interactively through an Open OnDemand GUI interface. A list of available applications can be found in Open OnDemand under Apps.

A session can be requested by selecting an application, e.g. Matlab:

ood_apps

This will take you to a request form where you will enter the resources needed for your job:

submit_form

Once your form is submitted, a tile will appear with job information and

queued active

Once its status has changed to active, you can connect and work interactively with your software.

Testing Your Knowledge

Batch Scripts

The original example script:

#!/bin/bash
#SBATCH --job-name=test
#SBATCH -e test.e%A
#SBATCH -o test.o%A
#SBATCH --partition=windfall
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --time=00:10:00
echo 'This script is running on:'
hostname
sleep 120

In the exercises below, try modifying and submitting the example submission script based on the prompts listed below:

For details on SLURM directives, see our SLURM documentation

✅ Challenge 1: Change your script to use 4 CPUs instead of 1.

Click here for the solution 👇


#!/bin/bash
#SBATCH --job-name=test 
#SBATCH -e test.e%A 
#SBATCH -o test.o%A 
#SBATCH --partition=windfall 
#SBATCH --nodes=1 
#SBATCH --ntasks=4 
#SBATCH --time=00:10:00 
echo 'This script is running on:' 
hostname 
sleep 120

✅ Challenge 2: Change your script to use the standard partition instead of windfall.

Click here for the solution 👇


#!/bin/bash
#SBATCH --job-name=test 
#SBATCH -e test.e%A 
#SBATCH -o test.o%A 
#SBATCH --partition=standard
#SBATCH --account=your_group
#SBATCH --nodes=1 
#SBATCH --ntasks=1
#SBATCH --time=00:10:00 
echo 'This script is running on:' 
hostname 
sleep 120

* If you don't know your group's name, run the command va to see which groups you are a member of.

✅ Challenge 3: Change your script so that it loads the system module python, version 3.6.5.

Click here for the solution 👇


#!/bin/bash
#SBATCH --job-name=test 
#SBATCH -e test.e%A 
#SBATCH -o test.o%A 
#SBATCH --partition=windfall
#SBATCH --nodes=1 
#SBATCH --ntasks=1
#SBATCH --time=00:10:00 
module load python/3.6/3.6.5

Getting Help

	Description	Link
Documentation	Detailed HPC information	docs.hpc.arizona.edu
Consulting Services	Contact us for HPC, statistical, or visualization consulting	Service Now request forms
Office Hours	Drop by for weekly virtual office hours, every Wednesday from 2:00-4:00 pm	Click here to join

Community Events

ResBaz

ResBaz provides a supportive community for anyone, of any level of expertise, involved in research computing. Weekly social events include Coffee & Code and Hacky Hour where you can go to meet other research professionals and get help. For more information, see: https://researchbazaar.arizona.edu/

Coffee and Code
Time: Tuesdays 8:00-10:00am. Location: Catalyst Café (Keating building), 1657 E Helen St.

Hacky Hour
Time: Thursdays 4:00-7:00pm. Location: Snakes & Lattes (Main Gate Square), 865 E University Blvd

UA Research Computing

HPC Examples

Intro to HPC Workshop

Contents

Overview

System Basics

Transferring Files

Submitting a Batch Job

Interactive vs. Batch

Accessing Software

Testing Your Knowledge

Getting Help

Community Events

Overview

System Basics

Logging In

Local terminal

Mac/Linux

Windows

Open OnDemand

Working on the Command Line

Linux Shell

HPC-Specific

Transferring Files

Submitting a Batch Job

Accessing Workshop Files

Submission Script Overview

Submitting Your Batch Job

Job Output

Interactive vs. Batch

Accessing Software

Modules on the Command Line

Software Through Open OnDemand

Testing Your Knowledge

Batch Scripts

Getting Help

Community Events