Create a container image for HPC


This KB outlines how to do the following:

  • Create a Docker image on your laptop
  • Export the image as a file and transfer it to a HPC server
  • Convert the image to an Apptainer (formerly Singularity) image
  • Run a container




Create a Python file with demo code

cat << 'EOF' >
import os, pickle, sys, time

CHECKPOINT_FILE = "fibonacci.pickle"

class Fibonacci:

  def __init__(self, n):
    self.checkpoint_i = 1
    self.n = n
    self.a = 0
    self.b = 1

  def run(self):
    print(f"Fibonacci: {self.n} (starting at calc {self.checkpoint_i})")
    self.checkpoint_time = time.time()
    if self.n == 0:
      return 0
    elif self.n == 1:
      return self.b
      for i in range(self.checkpoint_i, self.n):
        self.c = self.a + self.b
        self.a = self.b
        self.b = self.c
        if time.time() - self.checkpoint_time > 10:
          print(f"Checkpoint: {i}")
          self.checkpoint_i = i + 1
          with open(CHECKPOINT_FILE, 'wb') as f:
            pickle.dump(self, f)
          self.checkpoint_time = time.time()
      return self.b

# Handle calling Fibonacci from a script
if __name__ == "__main__":
  # allow for working with large numbers
  if os.path.exists(CHECKPOINT_FILE):
    with open(CHECKPOINT_FILE, 'rb') as f:
      fib = pickle.load(f)
    fib = Fibonacci(int(sys.argv[1]))
  # start/continue calculating
  result =
  print(f"Result: {result}")
  # clean-up checkpoint file if exists
  if os.path.exists(CHECKPOINT_FILE):

Run the Python file to test it

python ./ 30

Create a Dockerfile

cat << 'EOF' > Dockerfile
FROM python:3.10-alpine

COPY /src/

Build the image locally

If not on a Mac with a M1 chip

docker build -t fibonacci:latest .

If on a Mac with a M1 chip, build the image for x86_64 architecture

docker buildx build --platform linux/amd64 -t fibonacci:latest .

Run a container locally

docker run -it --rm fibonacci:latest python /src/ 30

Export the image

docker save fibonacci:latest -o fibonacci.tar

Transfer the image to Discovery

scp fibonacci.tar netid@discovery:

Build an Apptainer image from the Docker image

ssh netid@discovery

apptainer build fibonacci docker-archive://fibonacci.tar

Run an Apptainer container

apptainer exec fibonacci python /src/ 40

Create Slurm job script

cat << 'EOF' >
#SBATCH --job-name=fibonacci-apptainer
#SBATCH --error=%x-%J.err
#SBATCH --output=%x-%J.out
#SBATCH --requeue
#SBATCH --time=00:01:00
#SBATCH --mem=1G
##SBATCH --account=free
##SBATCH --partition=standard

scontrol show job $SLURM_JOBID > fibonacci-info.$SLURM_JOBID

stdbuf -i0 -o0 -e0 apptainer exec fibonacci python -u /src/ 3000001

Submit the job to Discovery


NOTE: You can use the squeue command to see the status of your job.

Print Article


Article ID: 153054
Fri 7/21/23 10:48 PM
Mon 9/25/23 9:47 AM