HPC Scratch Space

Scratch Space Described

Scratch space is temporary storage for files.  Most of the time you will want to use your own permanent storage but there are situations where scratch might be more appropriate.  For example,

  • You are doing a computation that requires large amounts of temporary space but only while the computation is running.
  • You have a data set larger than what you can fit into your own permanent storage but you only need it for less than 45 days.
  • You are doing a computation that needs higher performance storage than what your own permanent storage can provide (only applicable to local SSD/NVME scratch drives - not /scratch-nfs).

Scratch policies

  • We cannot make any guarantees about the safety of data in scratch volumes. It’s intended to be temporary and safety is traded away for performance. There are no backups and a disk failure usually means the permanent loss of all data in the volume.
  • Please don’t write directly to the top level of a scratch volume. Instead create a sub-directory that matches your username. For example “mkdir /scratch/mynetid”.
  • Files and directories older than 45 days in local scratch on discovery, polaris, or andes are automatically purged.  The same is true for /scratch-nfs.
  • Files and directories older than 20 days in local scratch on compute nodes are automatically purged.

Characteristics of specific scratch volumes

/dartfs-hpc/scratch aka /scratch-nfs

  • network storage, available to entire Discovery cluster plus Polaris and Andes
  • exact same performance as anything else mounted from DartFS (e.g. home and lab directories)
  • not subject to data loss from drive failure

computenode:/scratch

  • local to the node
  • newer nodes have SSD drives and performance will be noticeably better than /scratch-nfs for random I/O
  • older nodes have spinning disks and performance is probably around the same as /scratch-nfs
  • data loss with drive failure

polaris:/scratch

  • local to Polaris
  • a single NVME drive
  • best performance
  • data loss with drive failure

andes:/scratch

  • local to Andes
  • a single NVME drive
  • best performance
  • data loss with drive failure
Print Article