UA Research Computing

HPC Examples

Using an Anaconda Environment in a Batch Script

This example goes over the general process of creating and activating a local anaconda environment, installing packages, and using them in a batch job. For more information on anaconda on HPC, see our online documentation.

Creating your environment and installing packages

In this example, we’ll install some map-making software called eomaps into a conda environment. Once the software is installed, we’ll generate some fun maps in a batch job.

Start an interactive session

To get access to system software, you’ll need to be on a compute node. These can be accessed by requesting an interactive session using the command interactive from a login node:

💻 Desktop $ ssh
Last login: Wed Aug 24 14:31:24 2022
This is a bastion host used to access the rest of the RT/HPC environment.

Type "shell" to access the job submission hosts for all environments
[netid@gatekeeper 14:36:51 ~]$ shell
Last login: Wed Aug 24 14:26:30 2022 from
The default cluster for job submission is Puma
Shortcut commands change the target cluster
$ puma
(puma) $
$ ocelote
(ocelote) $
$ elgato
(elgato) $

(puma) [netid@wentletrap ~]$ elgato
(elgato) [netid@wentletrap ~]$ interactive -a hpcteam
Run "interactive -h for help customizing interactive use"
Submitting with /usr/local/bin/salloc --job-name=interactive --mem-per-cpu=4GB --nodes=1    --ntasks=1 --time=01:00:00 --account=hpcteam --partition=standard
salloc: Pending job allocation 375456
salloc: job 375456 queued and waiting for resources
salloc: job 375456 has been allocated resources
salloc: Granted job allocation 375456
salloc: Waiting for resource configuration
salloc: Nodes cpu4 are ready for job
[netid@cpu4 ~]$ 

Initialize Anaconda (if necessary)

If you’ve already initialized Anaconda in your account, you can ignore this step

Once your session starts, load the anaconda module including version:

[netid@cpu4 ~]$ module avail anaconda | grep anaconda
   anaconda/2020.02    anaconda/2020.11    anaconda/2022.05 (D)
[netid@cpu4 ~]$ module load anaconda/2022.05

If you have never used Anaconda in your account before, you’ll need to initialize it using conda init bash:

[netid@cpu4 ~]$ conda init bash
no change     /opt/ohpc/pub/apps/anaconda/2022.05/condabin/conda
no change     /opt/ohpc/pub/apps/anaconda/2022.05/bin/conda
no change     /opt/ohpc/pub/apps/anaconda/2022.05/bin/conda-env
no change     /opt/ohpc/pub/apps/anaconda/2022.05/bin/activate
no change     /opt/ohpc/pub/apps/anaconda/2022.05/bin/deactivate
no change     /opt/ohpc/pub/apps/anaconda/2022.05/etc/profile.d/
no change     /opt/ohpc/pub/apps/anaconda/2022.05/etc/fish/conf.d/
no change     /opt/ohpc/pub/apps/anaconda/2022.05/shell/condabin/Conda.psm1
no change     /opt/ohpc/pub/apps/anaconda/2022.05/shell/condabin/conda-hook.ps1
no change     /opt/ohpc/pub/apps/anaconda/2022.05/lib/python3.9/site-packages/xontrib/conda.xsh
no change     /opt/ohpc/pub/apps/anaconda/2022.05/etc/profile.d/conda.csh
modified      /home/u00/netid/.bashrc

==> For changes to take effect, close and re-open your current shell. <==

[netid@cpu4 ~]$ source ~/.bashrc  # <-- Make changes live. Run this instead of closing/reopening your shell
(base) [netid@cpu4 ~]$ 

Turn off auto-activiation feature

By default, Anaconda activates itself in your environment whenever you log in. This isn’t ideal and can result in a number of errors when using the system. To permanently prevent Anaconda from automatically activating:

[netid@cpu4 ~]$ conda config --set auto_activate_base false

Create your environment and install software

(base) [netid@cpu4 ~]$ conda create --name maps python=3.8
(base) [netid@cpu4 ~]$ conda activate maps
(maps) [netid@cpu4 ~]$ conda install -c conda-forge eomaps

Creating a batch job

The following steps can be taken either on a login node or on a compute node.

Python script

We’ll create a script based on an example from eomaps’ online documentation. We’ll make a directory in our home called maps-example and write the code to a file called

(elgato) [netid@wentletrap ~]$ cd maps-example
(elgato) [netid@wentletrap maps-example]$ ls

Where the contents of are:

#!/usr/bin/env python3

from eomaps import Maps, MapsGrid
mg = MapsGrid(crs=Maps.CRS.GOOGLE_MERCATOR)



# ... for more advanced
layer = mg.m_1_0.add_wms.ISRIC_SoilGrids.nitrogen.add_layer.nitrogen_0_5cm_mean
layer.set_extent_to_bbox() # set the extent according to the boundingBox                 # the "info" property provides useful information on the layer
layer()                    # call the layer to add it to the map
layer.add_legend()         # if a legend is provided, you can add it to the map!

SLURM script

Because conda’s auto-activate feature is turned off, we’ll need to use the command source ~/.bashrc to add conda to our environment:

#SBATCH --partition=standard
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --time=00:05:00

source ~/.bashrc  # <- Required to give you access to conda command if auto-activate has been turned off
conda activate maps

Submit the job

(elgato) [netid@wentletrap maps-example]$ ls  maps-example.slurm
(elgato) [netid@wentletrap maps-example]$ sbatch maps-example.slurm 
Submitted batch job 375475

Retrieve the results

A slurm-$JOBID.out file will be generated with all the text that would have been printed to the terminal had you run it interactively Additionally, you will receive a png file with the generated figure:

(elgato) [netid@wentletrap maps-example]$ ls
slurm-375475.out  grid.png  maps-example.slurm
(elgato) [netid@wentletrap maps-example]$ cat slurm-375475.out 
EOmaps: ... adding wms-layer OSM_default
EOmaps: ... adding wms-layer OSM_stamen_toner
EOmaps: ... adding wms-layer S1GBM_vv
EOmaps: fetching IRIS layers...
EOmaps: could not determine bbox from 'boundingBox'... defaulting to 'boundingBoxWGS84'

 LEGEND available: True

auth : <Authentication shared=False username=None password=None cert=None verify=True auth_delegate=None>
parent : < object at 0x2ac50242e730>
index : '1.4'
id : 'nitrogen_0-5cm_mean'
name : 'nitrogen_0-5cm_mean'
queryable : 1
title : 'Nitrogen 0-5cm mean (cg/kg)'
boundingBox : (-19949800.0,
boundingBoxWGS84 : (-179.998, -55.9773, 179.994, 82.7193)
crsOptions : ['EPSG:54012',
styles : {'default': {...}}
keywords : [None]
metadataUrls : [{...}]

EOmaps: ... adding wms-layer nitrogen_0-5cm_mean
Detailed performance metrics for this job will be available at by 8am on 2022/08/25.

To view the map, log into Open OnDemand, then:

  1. go to the Files dropdown tab

navigate to your maps-example directory, and click on grid.png