Skip to main content
Crusoe Support Help Center home page
Crusoe

How-To Update Image Pull Timeouts for Large Container Images on CMK

Apeksha Khilari
Apeksha Khilari
Updated

Introduction

If you're pulling large container images on a Crusoe Managed Kubernetes (CMK) cluster, you may see pulls fail partway through with an error like this from kubectl describe pod:

Failed to pull image "...": rpc error: code = Canceled desc = failed to pull and unpack image "...":
failed to extract layer (application/vnd.oci.image.layer.v1.tar+gzip sha256:...) to overlayfs as
"extract-...": context canceled

and this in the node's containerd logs around the same time:

level=error msg="cancel pulling image ... because of no progress in 5m0s"

ℹ️ Note: This is not specific to CCR (Crusoe Container Registry) — it can happen when pulling from any registry, upstream or otherwise. CCR is only responsible for delivering the layer's bytes over the network; extraction/unpacking to overlayfs happens entirely on the node afterward, and that's the phase this timeout actually trips on.

This is caused by containerd's CRI image_pull_progress_timeout setting, which defaults to 5 minutes. It's an idle-progress watchdog: if no new data is read from the registry for that long, containerd cancels the pull. The subtlety is that this same watchdog also covers the extraction phase, after the layer has finished downloading. Unpacking a large layer to overlayfs can itself take several minutes with zero new network bytes moving — which looks identical to a stalled pull from the watchdog's point of view, even though extraction is actively running.

The workaround is a DaemonSet that raises image_pull_progress_timeout to 15 minutes on every node so extraction of large layers has enough headroom to complete.

Prerequisites

  • Running CMK Cluster
  • CMK Nodes Running containerd 1.7.x or 2.x (CMK's Default Runtime)
  • kubectl Installed and Configured for the Cluster
  • Cluster-Admin Access to Deploy a Privileged DaemonSet to kube-system

Instructions

Step 1: Deploy the containerd Pull-Timeout DaemonSet

This DaemonSet runs on every node (including GPU and tainted nodepools, via its tolerations), detects your containerd version's CRI config section, patches image_pull_progress_timeout to 15m, and restarts containerd. It's idempotent — nodes already at 15m are skipped — and a restart of containerd does not affect already-running containers, since containerd supervises them through separate shim processes; only a pull that happens to be mid-flight at that exact moment would need to retry.

ℹ️ Note: This DaemonSet edits /etc/containerd/config.toml directly on the host and restarts the containerd service. It backs up the original file to /etc/containerd/config.toml.bak-pull-timeout before making any change.

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: containerd-pull-timeout
  namespace: kube-system
  labels:
    app: containerd-pull-timeout
spec:
  selector:
    matchLabels:
      app: containerd-pull-timeout
  template:
    metadata:
      labels:
        app: containerd-pull-timeout
    spec:
      # Run everywhere, including GPU/tainted nodes
      tolerations:
        - operator: Exists
      hostPID: true
      priorityClassName: system-node-critical
      containers:
        - name: patch
          image: ubuntu:24.04
          securityContext:
            privileged: true
          resources:
            requests:
              cpu: 5m
              memory: 16Mi
          command:
            - /bin/bash
            - -c
            - |
              set -euo pipefail
              CFG=/etc/containerd/config.toml
              WANT=15m
              NS="nsenter -t 1 -m -u -i -n --"

              if $NS grep -Eq "image_pull_progress_timeout *= *['\"]${WANT}['\"]" "$CFG"; then
                echo "image_pull_progress_timeout already ${WANT}, nothing to do"
              else
                $NS cp "$CFG" "${CFG}.bak-pull-timeout"

                if $NS grep -q image_pull_progress_timeout "$CFG"; then
                  # Default config dumps (containerd 1.7 and 2.x) already carry the key; replace its value
                  $NS sed -i -E "s/(image_pull_progress_timeout *= *)['\"][^'\"]+['\"]/\1'${WANT}'/" "$CFG"
                elif $NS grep -q "io.containerd.cri.v1.images" "$CFG"; then
                  # containerd 2.x section (cmk = 1.32 images)
                  $NS sed -i "/\[plugins\.'io\.containerd\.cri\.v1\.images'\]/a image_pull_progress_timeout = '${WANT}'" "$CFG"
                elif $NS grep -q "io.containerd.grpc.v1.cri" "$CFG"; then
                  # containerd 1.7.x section (older cmk images)
                  $NS sed -i "/\[plugins\.\"io\.containerd\.grpc\.v1\.cri\"\]/a image_pull_progress_timeout = \"${WANT}\"" "$CFG"
                else
                  echo "ERROR: no CRI section found in $CFG; leaving untouched" >&2
                  exit 1
                fi

                echo "patched $CFG - image_pull_progress_timeout = ${WANT}; restarting containerd"
                $NS systemctl restart containerd
              fi

              # Keep the pod alive so the DaemonSet stays Ready and covers nodes that join later
              sleep infinity

Step 2: Verify the Change Took Effect

Check the DaemonSet's logs to confirm each node was patched (or already compliant):

kubectl logs -n kube-system daemonset/containerd-pull-timeout

Optionally confirm directly on a node:

sudo containerd config dump | grep image_pull_progress_timeout

Resolution

Raising image_pull_progress_timeout to 15 minutes gives large-layer extraction enough headroom to finish before the watchdog cancels the pull, resolving the context canceled / no progress in 5m0s failure for most large-image cases.

This comes with a tradeoff worth knowing about: a genuinely stuck pull (a real network or registry problem, not just a large layer) now takes up to 15 minutes to fail and retry instead of 5, which can make actual problems slower to notice.

💡 Tip: If the same image is pulled repeatedly across many nodes or pods — a common pattern for training jobs that scale out — consider enabling Spegel, CMK's peer-to-peer image distribution. With Spegel enabled, only the first pull on the cluster needs to fetch the image from the registry (CCR or upstream); every subsequent pull on any other node is served from a peer that already has it. Since this timeout only affects pulls that actually go through the registry-and-extract path, Spegel effectively removes the problem for all pulls after the first.

Example

A team ships a 40 GB inference image whose largest layer is a 22 GB model bundle. Pulls from CCR download at full line rate, but the pod repeatedly lands in ImagePullBackOff with context canceled, and the node's containerd log shows no progress in 5m0s — the download finished, but unpacking the 22 GB layer to overlayfs took longer than the 5-minute watchdog allows.

They deploy the DaemonSet above, confirm each node logs either "patched" or "already 15m", and re-create the pod. Extraction now completes with headroom to spare, and because the DaemonSet stays Running, nodes added by the cluster autoscaler later get the same setting the moment they join.

Additional Resources

Related to

Was this article helpful?

0 out of 0 found this helpful

Still need help?

Our support team is ready to assist you with any questions.

Have more questions? Submit a request

Related Articles

Recently Viewed

Comments

0 comments

Article is closed for comments.