Skip to content

Slurm Basics

Slurm is RCC's workload manager.

What Slurm Does

Slurm:

  • Allocates access to compute nodes for jobs
  • Provides a framework for starting, executing, and monitoring jobs on allocated nodes
  • Manages the queue of pending work based on first come, first served order or predetermined scheduling rules

Do Not Run Jobs on the Head Node

The head node is special because it runs slurmctld, the central management daemon of Slurm. If the head node is overwhelmed, Slurm cannot do its job and computational resources become unavailable, including RStudio and Posit Workbench.

Do not compute on the head node

Do not run scripts or computational tasks on the head node. Submit work to Slurm instead.

Basic Commands

The Slurm cheat sheet is available at:

https://slurm.schedmd.com/pdfs/summary.pdf

sbatch

Submit a job script for later execution.

sbatch myScript.sh

Each submission is assigned a unique job ID.

Submitted batch job 30208

srun

Request interactive access to compute resources.

srun -p ccdefault -c 4 --mem-per-cpu=4G --pty /bin/bash

squeue

Report the state of active and pending jobs.

squeue
squeue -u USERNAME

scancel

Cancel a pending or running job.

scancel JOBID
scancel -u USERNAME

Helpful Commands

sacct

Get job accounting information about active or completed jobs.

sacct -j 28047 -o "jobid,jobname,reqcpus,reqmem,elapsed,state,exitcode" --unit=M

Example output:

JobID        JobName    ReqCPUS  ReqMem     Elapsed    State       ExitCode
------------ ---------- -------- ---------- ---------- ----------- --------
28047        filter_vc+ 2        8192M      00:02:01   COMPLETED   0:0
28047.batch  batch      2                   00:02:01   COMPLETED   0:0

This asks for job accounting information on job 28047 and reports:

  • Job ID
  • Job name
  • Requested CPUs
  • Requested memory
  • Time elapsed
  • Job state
  • Exit code

sinfo

Report the state of partitions and nodes managed by Slurm.

For example, sinfo can show whether nodes in ccdefault or highmem are idle, allocated, or mixed.

sstat and seff

These commands can report live resource use and job efficiency, but they were noted as not enabled in the original presentation.