Skip to content

Containerisation using Apptainer

BEAR Portal Apps

The apptainer command is not available inside several of the BEAR Portal apps, including JupyterLab, RStudio Server and BlueBEAR GUI. This is because these apps already run inside Apptainer containers.

BlueBEAR supports containerisation using Apptainer. Each node has Apptainer installed on its system, which means that the apptainer command is available without needing to first load a module.

Apptainer (previously Singularity1) is used instead of Docker on high performance computing systems due to the administrative privileges that are required to run the latter. However, Apptainer supports Docker images and can pull from Docker Hub. This means that in many cases it can act as a drop-in replacement for users who are already familiar with containerisation using Docker. Please see below for an outline of Apptainer operations.

Pull and Run

Apptainer can download images from any Open Container Initiative (OCI) repository and further information can be found here. For example, to pull an image from Docker Hub:

apptainer pull docker://python
apptainer pull docker://python:3.4.2 # (to pull a specific tag)

The two common methods for using an Apptainer container are exec and shell:

  • exec runs a command inside a container and then exits the container as soon as the command completes.
  • shell launches a container and then attaches to its shell.

/scratch directory size

Apptainer on BlueBEAR uses the /scratch directory for storing images (when they've not been explicitly pulled to another directory) and other container runtime data.
The size of /scratch varies between node-types and if container images are large then the volume can run out of space, which can cause errors. Our recommendation, if you encounter such issues, is to force your job to run on an Ice Lake node by including the following line in your SBATCH headers:

#SBATCH --constraint=icelake

Examples

apptainer exec python_3.4.2.sif python --version

will spawn a container from the specified image file, execute the command python --version, print the output and then exit the container.

Accessing Your Data

By default, Apptainer on BlueBEAR binds the following directories from the host node into each running container:

  • /rds: enables access to your home directories and project directories
  • /scratch: access to local disk storage (see here for further info)

Building Containers

--fakeroot not required!

If you have previous experience of building Singularity containers please note that it's no longer necessary to build using --fakeroot as the required privilege escalation is handled automatically.

To build an image from an Apptainer Definition File, please execute the following commands:

unset APPTAINER_BIND
apptainer build my_image.sif my_image_definition.def

Warning

If your image definition file includes software compilation then you will need to be aware of the node type on which you build the image, else you might have problems running the image.
Further general information on this can be found here: Self Installing Software for BlueBEAR

Interactive Development

To test development of an Apptainer image interactively use the --sandbox facility, which builds the image as a directory that can then be run with the --writable option.

Suggested workflow:

  1. Run: unset APPTAINER_BIND

  2. Create a sandbox directory either...

    1. from a base OS image, e.g. Rocky Linux:

      apptainer build --fix-perms --sandbox "/scratch/${USER}/my-sandbox-dir" docker://rockylinux:8.6

      or...

    2. From an Apptainer definition file:

      apptainer build --fix-perms --sandbox "/scratch/${USER}/my-sandbox-dir" ./my-definition-file.def

  3. Run the sandbox as a container in writeable mode with "root" privileges:

    apptainer shell --fakeroot --writable "/scratch/${USER}/my-sandbox-dir"

  4. Perform the necessary package installs and test your image's functionality iteratively.

  5. Write the required commands back into an Apptainer Definition File.
  6. Exit the sandbox container.
  7. Build the image from the resultant definition file according to the instructions above.

Build Apptainer images from Podman or Docker images

Apptainer sif images can be built directly from existing OCI Podman or Docker image files, when these are "saved" as tar files.

Note

The original images will need to have been built on a system matching the architecture of BlueBEAR, i.e. x86_64. This means that images built on Apple Silicon based Mac computers (e.g. M1, M2 etc.) will not work.

Process overview

  1. Use podman save or docker save on your local computer to export an already-built image to a tar file.
  2. Copy this tar file to BlueBEAR.
  3. Build the Apptainer image using the docker-archive bootstrap.

A detailed description of the process can be found in the Apptainer docs here:
https://apptainer.org/docs/user/latest/docker_and_oci.html#containers-in-docker-archive-files

Please also read the following information, which covers some of the limitations when running Podman/Docker images via Apptainer:
https://apptainer.org/docs/user/latest/docker_and_oci.html#differences-and-limitations-vs-docker

Docker Desktop licensing

Note that if you're using the Docker Desktop software on your local computer, you may require a licence.

The Docker Desktop software for Windows and macOS systems has a restricted licence that does not fully-exempt educational environments.
Section 4.2 of the Docker Subscription Service Agreement contains the following definitions:

(a) The Docker Desktop component of the Service at the level of the Personal Offering (as described on the Pricing Page) is further restricted to: (i) your “Personal Use”, (ii) your “Educational Use”, (iii) your use for a non-commercial open source project, and (iv) your use in a “Small Business Environment”.

(b) For purposes of this Section 4.2: (i) “Personal Use” is the use by an individual developer for personal use to develop free or paid applications, (ii) “Educational Use” is the use by members of an educational organization in a classroom learning environment for academic or research purposes or contribution to an open source project [...]

Alternatives to Docker Desktop:

  1. The Apptainer application replaces the previous HPC containerisation solution, Singularity. It is broadly a drop-in replacement and for now the singularity command is still available, although it will actually execute the Apptainer command.
    For further information on the move from Singularity to Apptainer please see this article. ↩