How To Update the IB Partition on a VM using the Crusoe CLI

Introduction

There may be scenarios where you need to change the InfiniBand (IB) network of a Virtual Machine (VM) due to capacity constraints in one IB network and available capacity in another.

Prerequisites

Before proceeding, ensure the following:

• You have active access to the Crusoe Cloud Console.

• The Crusoe CLI is installed. (Note: Changing the IB network can currently only be done via CLI.)

• IB partitions have been created for the target InfiniBand networks.

• The VM must be stopped before updating its IB network. Please ensure you backup any ephemeral data on the VM before stopping.

• IB partitions can only be updated within the same region.

Step-by-Step Instructions

Step 1: Stop the VM

For the purpose of this guide, I have a VM running in the following InfiniBand (IB) network:

• Region: us-east1-a

• Instance Type: H100-80GB SXM

• IB Network: IB Network 3

• IB Network ID: b049a2cf-63b1-4cfd-9e2d-b1541d87b538

• IB Partition ID: 87f2e036-7940-4af0-80e9-8bec61abbd02

crusoe compute vms list --name rgee-test-h100
name                                  type                 location              state        
----------------------------------------------------------------------------------------------
rgee-test-h100                        h100-80gb-sxm-ib.8x  us-east1-a            STATE_RUNNING

crusoe compute vms list --name rgee-test-h100 --format json
[
    {
        "billing_type": "on-demand",
        "created_at": "2025-07-24T17:25:34Z",
...
...
        ],
        "host_channel_adapters": [
            {
                "guid": "41b0:0820:e77c:f41e",
                "ib_network_id": "b049a2cf-63b1-4cfd-9e2d-b1541d87b538",
                "ib_partition_id": "87f2e036-7940-4af0-80e9-8bec61abbd02",
                "type": "infiniband"
            },

After stopping and attempting to restart the VM above, the underlying node type is now out of stock.

Step 2: Locate Available Capacity

You can view available capacity by navigating to the "Networking" tab in the Crusoe Cloud Console, then selecting "InfiniBand."
The Capacity column on the right side of the page will display the current availability for each IB network.

At the time of writing, there is available capacity in us-east1-a on the H100-80GB SXM IB Network 5 (474dbfe2-ffb3-44ce-87a4-cea659fddea3), with 16 GPUs currently available.

Note: You can also check GPU Capacity from the API following this doc here

Step 3: Update the IB partition on a VM using the Crusoe CLI

I can update the VM’s partition ID to the new partition associated with us-east1-a H100-80GB SXM IB Network 5. The new partition ID is: 97da27ea-c1e7-43a7-9a35-7a9af9c5686a.

Note: The VM must be stopped before updating the partition ID.

Run this command from the Crusoe CLI:

crusoe compute vms update rgee-test-h100 --ib-partition-id 97da27ea-c1e7-43a7-9a35-7a9af9c5686a
name:           rgee-test-h100
id:             47eee5d9-aec0-47e0-8079-9a27f660a6cf
type:           h100-80gb-sxm-ib.8x
state:          STATE_SHUTOFF
public IP:      not assigned
private IP:     172.27.26.184
internal DNS:   rgee-test-h100.us-east1-a.compute.internal
external DNS:
created at:     2025-07-24T17:25:34Z
reservation:    --

After updating and viewing the VM, I can confirm that both the IB Network ID and IB Partition ID have successfully changed to those associated with us-east1-a H100-80GB SXM IB Network 5.

crusoe compute vms list --name rgee-test-h100 --format json
[
	{
		"billing_type": "on-demand",
		"created_at": "2025-07-24T17:25:34Z",
		"disks": [
...
...
		],
		"host_channel_adapters": [
			{
				"guid": "",
				"ib_network_id": "474dbfe2-ffb3-44ce-87a4-cea659fddea3",
				"ib_partition_id": "97da27ea-c1e7-43a7-9a35-7a9af9c5686a",
				"type": "infiniband"
			},

Step 4: Start the VM

You can now start the VM using either the Crusoe Cloud Console or the Crusoe CLI to utilize the available compute capacity.

If you are still observing out of stock errors, please contact support@crusoe.ai

Additional Resources

1. Managing-infiniband-networks
2. Check GPU Capacity from the API