FAQ: Using Enroot and Pyxis on Slurm Clusters

Last Updated: March 30th, 2026

Introduction

Enroot and Pyxis let you run containerized workloads on a Slurm-managed HPC cluster. These tools work together so that users can launch containers into scheduled jobs.

Question 1: What is Enroot?

Enroot is a lightweight container runtime optimized for HPC environments. Converts Docker or OCI images into lightweight, runnable containers. 

• Allows users to run containers without root privileges. 

• Faster and lighter than traditional Docker, with minimal overhead. 

• Provides process and filesystem isolation suitable for high-performance workloads.

• Instead of installing software directly on the cluster, you can use pre-built containers that include CUDA or other dependencies.

# Check Enroot version
$ enroot version
4.1.1

# Import a Docker image
$ enroot import docker://nvcr.io#nvidia/pytorch:23.10-py3
...
[INFO] Creating squashfs filesystem...
Parallel mksquashfs: Using 128 processors
Creating 4.0 filesystem on /home/ubuntu/nvidia+pytorch+23.10-py3.sqsh, block size 131072.

# Optional step to create the container from the existing .sqsh file
$ enroot create nvidia+pytorch+23.10-py3.sqsh
[INFO] Extracting squashfs filesystem...

# List container
$ enroot list
nvidia+pytorch+23.10-py3

# Start the container
$ enroot start nvidia+pytorch+23.10-py3 \
python3 -c "import torch; print(f'CUDA Version: {torch.version.cuda}')"
=============
== PyTorch ==
=============
NVIDIA Release 23.10 (build 71412639)
PyTorch Version 2.1.0a0+32f93b1
...
CUDA Version: 12.2

Question 2: What is Pyxis?

Pyxis is a plugin for Slurm, which uses Enroot to allow cluster users to run containerized jobs by using the srun command with additional --container-*** parameters.

• Accepts --container-image in Slurm commands. 
• Automatically imports Docker images (enroot import). 
• Creates containers (enroot create) and starts containers (enroot start) on the allocated compute node. Eliminates the need to manually copy .sqsh files or create containers. 
• Jobs respect Slurm resource allocations (CPUs, GPUs, memory) just like native jobs.

$ srun --container-image=docker://nvcr.io#nvidia/pytorch:23.10-py3 \
  python3 -c "import torch; print(f'CUDA Version: {torch.version.cuda}')"
pyxis: importing docker image: docker://nvcr.io#nvidia/pytorch:23.10-py3
pyxis: imported docker image: docker://nvcr.io#nvidia/pytorch:23.10-py3

CUDA Version: 12.2

Additional Resources

Enroot - GitHub

Pyxis - GitHub