Table of Contents

High Performance Computing (HPC) can appear daunting at first. Understanding a few basic fundamentals can go a long ways to empowering you to perform complex computational analysis in order to propel your research forward. The biggest challenge in learning HPC is knowing where to get started.

Most HPC environments require a fairly basic knowledge of Linux/Unix command line and some scripting knowledge. Most HPC environments use Linux for the operating system. Starting out, we will assume that you have some knowledge of command line arguments. A shell scripting language, such as BASH/CSH/KSH is also beneficial and can augment the capabilities of batch submission scripts you create.

Mailing List

If you or your students are using Aeolus cluster (all users with an account or access), you and/or they should be subscribed to the Aeolus mailing list.

This list is for ALL announcements and is required to be signed up to. The Aeolus mailing list distributes emails that cover important developments concerning the cluster resources and operations; you need to know about these things.

Support

Aeolus has its own support website for ALL Aeolus issues and requests:

  • If you want to setup an account on Aeolus
  • If you have a question or need to report an issue
  • If you need help installing a software package

Getting Connected

Account Request(s)

To create an account, follow the guide found here.

Once you have been confirmed an account on the system, you will have access to the cluster and can work through the documentation for examples of using the system and submitting jobs.

Linux

Linux provides many terminal environments. Additionally, SSH is a de-facto standard for interfacing remotely with Linux systems and as a result, it is usually installed already. To login, use the following procedure:

ssh [username]@aeolus.wsu.edu

Mac

Mac has a Terminal application provided by OSX. No additional software is required to connect. To login, use the following procedure:

ssh [username]@aeolus.wsu.edu

Windows

Windows users will require additional tools to be installed in order to access Aeolus via SSH. Common tools include Putty, which is a windows based SSH program.

For those of you using Putty, you don’t have to live in the misery that is the basic configuration of colors. As a personal favourite, we would recommend the monokai theme, which can be found on GitHub.

Keys (password-less)

Passwords provide one level of security to authenticating into the system. If you need to automate tasks and/or want a more secure and easier way of accessing the system, you should use a SSH key.

For now, please consult this tutorial on how to do just that. Consider the following notes:

  1. Aeolus is a legacy system that doesn't support the Ed25519 algorithm, you must use RSA algorithm as described in the tutorial.
  2. Do not leave the file name “id_rsa”, that is too generic. Name it something meaningful, like "rsa4096l_amb_wsu"
    • rsa4096 = encryption type
    • l = legacy systems
    • amb = initials
    • wsu = systems that will be connecting
  3. Definitely provide a passphrase to the key! If you do not, you will still be held responsible for all actions taken on your account, even if your key gets compromised.
    • The passphrase has nothing to do with your password on the system you are connecting to. It is used to protect the encrypted key on your system.
  4. Add it to your ssh-agent, that way you don’t have to unlock the key every time you want to log in (just per your local sessions).

Useful Linux Commands

There are a plethora of commands out there for Linux, which is both a positive and a negative. We want to get you pointed in the right direction with some basic command line interface (CLI) commands:

NOTE: for all of the example commands that we will cover here, consider the following variable nomenclature:

  • [Required]
  • <Optional>

ANY and ALL questions about commands should first refer you to the Linux Manual (man pages), which gives us our first command:

man man
man [some other cli command]

Checking Storage

For information about storage options, visit this guide.

Disk Usage (du)

  • du [/path/to/dir] total disk useage of directory
  • du -k [/path/to/dir] total disk useage of directory in k bytes
  • du -sc --si [/path/to/dir] summarize, add up total, in powers of 1024
  • du -sc --si [/path/to/dir]/* | sort -h same as above, and sort it by human readable numbers

Report Filesystem Usage (df)

  • df -h [/path/to/filesystem] print as human readable

Pattern Matching (grep)

  • grep -i [pattern] [path] search for pattern
  • grep -v [pattern] [path] ignore pattern
  • grep -R [pattern] [path] recursively search for pattern

Data Transfer

For a more thorough overview on how to utilize rsync, please review the specific Knowledge Base Article

Cron & Crontab

Cron is a Unix utility that allows tasks to be automatically run in the background at regular intervals by the ssh-server cron daemon.

Backup your cron file:


mkdir ~/mycron crontab -l > ~/mycron/'crontab.$(date +%Y%m%d)'.backup



WARNING: crontab -e and crontab -r are grossly different commands, yet e & r appear side by side on a standard QWERTY keyboard… thus, learning to back it up first!

  • crontab -e edit or create your crontab file using your default editor
  • crontab -l display your crontab file
  • crontab -v display the last time you edited your crontab file
  • crontab -r remove your crontab file

For an example base cron file & help:


# m h dom mon dow command 
# * * * * * command to be executed 
# - - - - - 
# | | | | | 
# | | | | +----- day of week (0 - 6) (Sunday=0) 
# | | | +------- month (1 - 12) 
# | | +--------- day of month (1 - 31) 
# | +----------- hour (0 - 23) 
# +------------ min (0 - 59)

Crontab Examples:

  • 30 18 * * * rm ~/tmp/* remove all files (that you have access to) from ~/tmp/, each day at 6:30 pm
  • 0 9,18 * * Mon ~/scripts/example.sh run ~/scripts/example.sh every Monday at 9:00 am and 6:00 pm

Other/External Tutorials

Don’t just take our word for it, do some research on your own! Always feel free to come back and ask us questions by submitting a ticket.

Here are some extra tutorials to arm yourself for familiarity with Linux:

Job Submission

Moving beyond the base Linux system commands & navigation, to actually submit a job will require becoming familiar with TORQUE & MAUI.

For more detailed information and examples of submitting jobs on Aeolus, please see the Job Submission documentation.