NCAR HPC Environment#

2023 CESM in person tutorial specific instructions#

Tutorial Project Account#

You should have access to project account UESM0012 and use this for your simulations.

Special Queues#

We have a special queue every day for running to ensure you get through the Cheyenne queues quickly and get your jobs run. These are only active for portions of each day during our lab sessions and change for each session. So you should be sure you are using the correct reservation queue.

Schedule for Tutorial Reservation Queues

Mon      14:00 - 17:00    cheyenne   R1787445
Tue      10:30 - 12:00    cheyenne   R1787503
         14:00 - 17:00    cheyenne   R1787518
Wed      10:30 - 12:00    cheyenne   R1787531
         14:00 - 17:00    cheyenne   R1787589
Thu      13:30 - 17:00    casper     R7873845
Fri      13:30 - 17:00    cheyenne   R1787669

Using these tutorial-specific codes#

In theory, you should be able to automatically use this account and the reservation queues if you correctly set up your NCAR HPC environment using your .profile or .tcshrc files, as described below. If you need to change the values for a run, you can change the account and reservation queue in your case directory using the following command:

./xmlchange --force JOB_QUEUE=R1787445
./xmlchange PROJECT=UESM0012

For instance, on Wednesday

  • If (Wed 10:30 - 12:00) then

./xmlchange JOB_QUEUE=R1787531 --force

  • If (Wed 14:00 - 17:00) then

./xmlchange JOB_QUEUE=R1787589 --force

  • If (Wed at another time) then

./xmlchange JOB_QUEUE=regular

If you do use these commands, make sure for the JOB_QUEUE you are using the correct reservation code for the relevant day and time of that lab session. You should not have to set the PROJECT more than once during the tutorial if your profile is set up, as described below, but we include it for your reference in case you are having trouble.

Setting up your NCAR HPC environment#

Every time you log onto the NCAR HPC you want to ensure you have the correct modules loaded.

NOTE: The next section describes how to use a file that loads the modules necessary for those attending the in-person tutorial. Once you set up your environment you should not have to do it again for the remainder of the tutorial. Every time you login to NCAR HPC these files will automatically be sourced. However, if you intend to use NCAR HPC after the tutorial you may need to modify your `.profile` or `.tcshrc` file to load the correct modules, project numbers, or queues.

SHELL environment#

To determine what shell environment you are using on the NCAR HPC, type the following command:

echo $SHELL

If the command returns, you are a bash user /bin/bash

If the command returns, you are a cshell user /bin/tcsh

NOTE: All new NCAR HPC accounts default to bash.

Bash users#

NOTE: You may already have a .profile file in your home directory.

  • If you have an existing .profile file and do not wish to overwrite its contents, please save your original .profile file to a new filename before copying the tutorial .profile file into your home directory.

  • Alternatively, you could copy the contents of the tutorial profile file into your existing .profile file using a text editor. (see special instructions below)

One-time setup#

To set up your environment for the tutorial, follow the following three steps:

Copy over the .profile file into your home directory:

cp /glade/p/cesm/tutorial/profile ~/.profile

Source the file:

source ~/.profile

Before each practical#

Do the following step before every practical. This is because we are getting a different special queue every day.

Source the file:

source ~/.profile

Tcsh users#

NOTE: You should already have a .tcshrc file in your home directory. If you do not wish to overwrite the existing .tcshrc file, please save this file to a new name OR copy the contents of the /glade/p/cesm/tutorial/tcshrc file to your .tcshrc file using a text editor.

One-time setup#

To set up your environment for the tutorial, follow the following three steps:


Copy over the .tcshrc file to your home directory:

cp /glade/p/cesm/tutorial/tcshrc ~/.tcshrc

Source the file:

source ~/.tcshrc

Before each practical#

Do the following step before every practical. This is because we are getting a different special queue every day.

Source the file:

source ~/.tcshrc

If you have any trouble#

Please ask for help if you are having trouble with this step!

The modules necessary for this tutorial and loaded through these files are:

  • ncarenv/1.3

  • intel

  • ncl

  • nco

  • ncview

  • netcdf

Special instructions (if you already had an account on cheyenne)#

If you already have a .profile or .tcsh and you do not wish to overwrite its contents, you need to add the following into your existing .profile or .tcsh using a text editor.

Bash:#

Add to your .profile:

export PROJECT=UESM0012         #<- This should never change over the course of the tutorial
export TUTORIAL_QUEUE=R1787445  #<- This will be different for every practical. 
                                #<- Look at the schedule below:

Source the file:

source ~/.profile

C-shell:#

Add to your .tcshrc:

setenv PROJECT UESM0012         #<-This should never change over the course of the tutorial
setenv TUTORIAL_QUEUE R1787445  #<-This will be different for every practical 
                                #<- Look at the schedule below:

Source the file:

source ~/.tcshrc